You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: tasks/PickAndPlaceChallenge.tex
+53-64Lines changed: 53 additions & 64 deletions
Original file line number
Diff line number
Diff line change
@@ -1,22 +1,25 @@
1
1
\section{Pick and Place Challenge}
2
2
\label{test:pick-and-place-challenge}
3
-
The robot tidies up the kitchen table after a dinner.
4
-
The tableware and cutlery must be placed inside the dishwasher.
5
-
The snacks must be placed inside the trash bin.
6
-
All other objects must be stored in a cabinet with shelves. Objects are sorted on the shelves based on similarity, for instance, an apple is stored next to other fruits.
7
-
The robot has to set the table for breakfast for one person.\\
8
-
9
-
\noindent\textbf{Main goals:}
10
-
Tidy up all the objects on the dinner table. Tableware and cutlery have to be placed correctly inside the dishwasher, snacks have to be placed inside the trash bin and all other objects have to be placed in the cabinet, grouping them by category or similarity.
11
-
Place the breakfast items on a table (bowl, spoon, cereal box, and milk). \\
3
+
This challenge evaluates manipulation capabilities through cleaning and organizing the kitchen and preparing a simple breakfast.\\
12
4
5
+
\noindent\textbf{Main goals:}
6
+
\begin{enumerate}[nosep]
7
+
\item Tidy up all objects on the dining table:
8
+
\begin{enumerate}[nosep]
9
+
\item Place dirty tableware and cutlery inside the dishwasher.
10
+
\item Place designated trash items in the trash bin.
11
+
\item Store other objects in the cabinet, grouping them with similar items.
12
+
\end{enumerate}
13
+
\item Set up breakfast on a clean area of the table, including a bowl, spoon, cereal, and milk.
14
+
\end{enumerate}
15
+
\smallskip
13
16
\noindent\textbf{Optional goals:}
14
17
\begin{enumerate}[nosep]
15
-
\itemOpening and closing the dishwasher door
16
-
\itemPulling and pushing the dishwasher rack
17
-
\itemPlacing a dishwasher tab inside the dishwasher
18
-
\itemPicking objects from a shopping bag
19
-
\item Pour milk and cereal into the breakfast bowl
18
+
\itemPick up trash from the floor.
19
+
\itemOpen and close the dishwasher door.
20
+
\itemPull and push the dishwasher rack.
21
+
\itemPlace a dishwasher tablet inside the dishwasher.
22
+
\item Pour milk and cereal into the breakfast bowl.
20
23
\end{enumerate}
21
24
22
25
\subsection*{Focus}
@@ -29,39 +32,37 @@ \subsection*{Setup}
29
32
\begin{itemize}[nosep]
30
33
\item\textbf{Locations:}
31
34
\begin{itemize}
32
-
\item\textbf{Start Location:} Before the test, the robot waits outside the Arena and navigates to the testing area when the door is open.
33
-
\item\textbf{Test location:} This test takes place in the kitchen.
35
+
\item\textbf{Start Location:} The robot waits outside the arena and enters when the door is opened.
36
+
\item\textbf{Test location:} The test takes place in the kitchen.
34
37
\end{itemize}
35
38
\item\textbf{People:}
36
39
\begin{itemize}
37
-
\item No people are involved in the test, unless the robot requires human assistance.
40
+
\item No people are involved unless the robot explicitly requests assistance.
38
41
\end{itemize}
39
42
\item\textbf{Furniture:}
40
43
\begin{itemize}
41
-
\item\textbf{Dishwasher:} A dishwasher is located close to the dining table. The dishwasher is closed by default, any help the robot may need to open or close the doors or racks must be requested by the robot during the task.
44
+
\item\textbf{Dishwasher:} Located near the dining table. Closed by default; the robot may request help to open or close doors or racks.
42
45
\item\textbf{Trash bin:} A trash bin is located in the kitchen.
43
-
\item\textbf{Shopping Bag:} There is a shopping bag on the ground next to the table.
44
-
\item\textbf{Cabinet:} The cabinet contains objects arranged in groups, either by category or likeness, on different shelves.
46
+
\item\textbf{Cabinet:} The cabinet contains objects arranged by category or similarity on different shelves.
45
47
\end{itemize}
46
48
\item\textbf{Objects:}
47
49
\begin{itemize}
48
-
\item\textbf{Table setting:} The table has a total of seven objects arranged arbitrarily or in a typical setting after a meal. The cutlery and tableware objects may be randomly stacked as is common after a meal.
49
-
The object distribution is as follows:
50
+
\item\textbf{Table setting:} Seven objects arranged on the table in a typical post-meal setting, possibly stacked:
50
51
\begin{itemize}[nosep]
51
-
\item\textit{Cutlery}: One piece of cutlery (fork, knife, or spoon).
52
-
\item\textit{Tableware}: Any two objects (except cutlery), at least one of which is a plate (cup, plate).
53
-
\item\textit{Snacks}: One snack from the object set.
54
-
\item\textit{Other objects}: Any three additional objects from the object set, excluding cutlery, tableware, and snacks.
52
+
\item\textit{Cutlery}: One piece (fork, knife, or spoon).
53
+
\item\textit{Tableware}: One mug or cup and one plate.
54
+
\item\textit{Trash}: One trash item. One object category will be treated as trash for this task.
55
+
\item\textit{Other objects}: Two items not belonging to the above categories.
55
56
\end{itemize}
56
-
\item\textbf{Shopping Bag:} Any three additional objects from the object set, excluding cutlery, tableware, and snacks.
57
57
\item\textbf{Breakfast items:} The breakfast items are a bowl, a spoon, milk and cereal.
58
58
The object distribution is as follows:
59
59
\begin{itemize}[nosep]
60
-
\item\textit{Bowl and Spoon}: On top of the dishwasher.
61
-
\item\textit{Milk and Cereal}: Inside the cabinet, next to their respective category.
60
+
\item\textit{Bowl and Spoon}: On top a designated surface in the kitchen.
61
+
\item\textit{Milk and Cereal}: Inside the cabinet, next to their respective categories.
62
62
\end{itemize}
63
-
\item\textbf{Cabinet objects:} The objects are placed in a doorless cabinet. Each side of a shelf contains objects arranged in groups, either by category or likeliness.
64
-
\item\textbf{Dishwasher tab:} The tab can be found on top of the dishwasher and should be placed inside the tab slot in the dishwasher.
63
+
\item\textbf{Cabinet objects:} A doorless cabinet. Each side of the shelves contains objects arranged in groups, either by category or likeliness.
64
+
\item\textbf{Dishwasher tab:} The tab can be found on top of a designated surface and should be placed inside the dishwasher slot.
65
+
\item\textbf{Floor Object:} One trash item will be placed near the trash bin.
65
66
\end{itemize}
66
67
\end{itemize}
67
68
@@ -72,8 +73,8 @@ \subsection*{Setup}
72
73
\subsection*{Procedure}
73
74
\begin{enumerate}[nosep]
74
75
\item\textbf{Test start:} The robot moves to the kitchen when the arena door is open.
75
-
\item\textbf{Table clean up:} The robot tidies up the table by putting: the cutlery and tableware items inside the dishwasher, the snack in the trash bin and the other objects in the cabinet, grouping them by category or similarity.
76
-
\item\textbf{Serve breakfast:} The robot sets the table for breakfast by placing the bowl, spoon, cereal and milk on the table in a typical setting for a meal.
76
+
\item\textbf{Table clean up:} The robot tidies up the table by putting: the cutlery and tableware items inside the dishwasher, the trash in the trash bin and the other objects in the cabinet, grouping them by category or similarity.
77
+
\item\textbf{Serve breakfast:} The robot sets the table for breakfast by placing the bowl, spoon, cereal and milk on the table in a typical setting for a meal. There needs to be a comfortable amount of free space around the breakfast.
77
78
\item\textbf{Sequence:} The robot is free to determine the order and method for performing the pick-and-place tasks. There is no predefined sequence, the robot may execute them in any way it finds optimal.
\item\textbf{First Pick Bonus:} To encourage manipulation, the robot receives an additional bonus for successfully picking the first object during the test. This bonus is awarded only once.
88
+
\item\textbf{Designated Location:} All objects designated location correspond to the appropriate furniture, either on top of or inside.
86
89
\item\textbf{Safe placing:} Objects must be placed with care, namely the robot should place rather than throw or drop objects.
87
90
\item\textbf{Dishwasher door:} The dishwasher door is closed by default.
88
91
The robot may ask for help to open or close the door or racks at any time during the task. If the robot fails to open/close the door/rack, it must clearly state this and request the referee to open/close the door/rack.
89
-
\item\textbf{Correct dishwasher item placement:} The items should be placed in the rack correctly, namely in the location and in a manner as a person would put them.
90
-
\item\textbf{Incorrect cabinet category categorization:} The score is reduced if an object is stored in the cabinet, but not next to similar objects; this reduction is applied per incorrectly stored object.
91
-
\item\textbf{New category:} Objects that do not semantically belong to any of the categories represented on the shelves should be grouped together in an empty part of the shelf.
92
-
\item\textbf{Correct serve breakfast item placement:} The table must be set in a typical setting for a meal. The spoon must be placed next to the bowl, and the cereal and milk must be placed next to each other.
92
+
\item\textbf{Correct dishwasher item placement:} Items must be correctly positioned in the rack, as a human would place them.
93
+
\item\textbf{Incorrect cabinet category categorization:} Objects must be grouped with similar items. Misplaced items incur score reductions. Objects that do not semantically belong to any of the categories represented on the shelves should put in an empty part of the shelf.
94
+
\item\textbf{Breakfast placement:} The table must be set in a typical setting for a meal. The spoon must be placed next to the bowl, and the cereal and milk must be placed next to each other.
95
+
\item\textbf{Breakfast area cleanliness:} The area immediately surrounding the breakfast items on the table must be kept clear of any other objects. Items too close at (5cm) or cluttering the space, will result in a score penalty for the breakfast placement.
93
96
\item\textbf{Pouring:} A significant amount of the milk and cereal must be poured, Pouring a couple of drops of milk or bits of cereal is not enough.
94
-
\item\textbf{Deus ex Machina:} The scores are reduced if human assistance is received, in particular:
95
-
\begin{itemize}[nosep]
96
-
\item Pointing to an object or telling the robot where an object is
97
-
\item Telling or pointing out to the robot where to place an object
98
-
\item Handing an object over to the robot
99
-
\item Moving an object instead of the robot
100
-
\item Changes to the environment that aid the robot in completing the task (e.g remove chairs or decorations from dinner table).
101
-
This penalty is applied per group of changes. Chairs are one group, decorations are a different group. (e.g One penalty is applied to remove all chairs. The penalty is not applied per chair removed)
102
-
\end{itemize}
103
-
\item\textbf{Communicating Perception}: The robot must clearly communicate its perception to the referee.
104
-
Pointing at the object or attempting to pick it up is sufficient. When using visualization, make sure the robot
105
-
tells the referee where to look and makes the visualization easily accessible.
106
-
If the team wants to utilize bounding boxes make sure \textbf{only} one object with a bounding box is shown
107
-
at a time so the referee is easily able to check and verify. Also make sure the surrounding scene is visible, i.e.
108
-
just showing a cropped bounding box is not enough.
97
+
\item\textbf{Human Assistance:} Scores are reduced if the robot receives help, such as pointing to objects, handing objects to the robot, or repositioning items. Assistance with opening the milk container or moving parts of the dishwasher does not incur a penalty.
98
+
\item\textbf{Communicating Perception}: The robot must clearly indicate its perception to the referee. Pointing, attempting to pick objects, or visualizing one object at a time is sufficient. If visualization is utilized, the surrounding scene must remain visible and the robot needs to announce and confirm the referee perceived the visualization.
99
+
\item\textbf{Trash:} One object category will be treated as trash for this task. Announced during \SetupDays{}.
109
100
\end{enumerate}
110
101
111
102
\subsection*{OC Instructions}
112
103
113
104
During the \SetupDays:
114
105
\begin{itemize}
115
-
\item Provide official cutlery and tableware for training.
116
-
\item Provide official objects for training.
117
-
\end{itemize}
118
-
At least two hours before the test:
119
-
\begin{itemize}
120
-
\item Announce which table and cabinet will be used in the test.
121
-
\item Select which bags will be used in the test.
106
+
\item Provide official cutlery and tableware.
107
+
\item Provide official objects.
108
+
\item Designate a trash category.
109
+
\item Announce the table and cabinet used for the test.
110
+
\item Announce locations of dishwasher tablet, bowl, and spoon. (in the kitchen)
122
111
\end{itemize}
123
112
124
113
125
114
\subsection*{Referee Instructions}
126
115
127
116
The referee needs to:
128
117
\begin{itemize}
129
-
\item Place objects on the table (one piece of cutlery, two pieces of tableware, one snack, and three other objects).
130
-
\item Place objects in the shopping bag (three objects).
131
-
\itemPlace objects in the cabinet, grouping them by category or likeliness.
132
-
\item Place the bowl and spoon on top of the dishwasher.
118
+
\item Place table objects (1 cutlery, 1 plate, 1 mug or bowl, 1 trash, 2 other objects).
119
+
\item Place one trash on the floor (near trash bin).
120
+
\itemArrange cabinet objects by category or similarity.
121
+
\item Place the bowl and spoon and dishwasher tab.
133
122
\item Place the milk and cereal inside the cabinet next to their respective category.
0 commit comments