wrs2020 test log

SCOREBOARD

	Taskboard	Assembly
5.2.2021	24 (44)	-
12.2.2021	0	-
19.2.2021	20(44)	-
26.2.2021	34	-
5.3.2021	34	-
12.3.2021	10(20)	-
19.3.2021	34	-
26.3.2021	24	-
2.4.2021	38	-
9.4.2021	28	-
13+16.4.2021	14	0(4)
20+23.4.2021	0(44)	0(4)
27+30.4.2021	34	0(4)
7.5.2021	34
11+14.5.2021	28	0(2)
18+21.5.2021	58	0
25+28.5.2021	10	0
01+04.6.2021	108	0
08+11.6.2021	62	0
15+18.6.2021	48	0(4)
22+25.6.2021	86	0
29.6.+2.7.2021	34	0(7)
6+9.7.2021	44	0
13+16.7.2021	58	7
21+23.7.2021	--	5
27+30.7.2021	58	0
3+6.8.2021	****	4
10+13.8.2021	84	8
17+20.8.2021	****	10
24+27.8.2021	****	11
31.8+3.9.2021	120	8

Test Logs

3.9.2021 (Taskboard)

Subtask	Score
Set screw (14)	O
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	O
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	O

Time bonus	-
TOTAL	120(20 time bonus)

=== PROBLEMS / THOUGHTS

Belt picked recognition failed a few times, wait for the motion to be done before checking belt
Screw retreat and approach can be faster

27.8.2021 (Taskboard)

Reset 17:00, camera disconnected. Restart 14:10
Finish at 3:20

Subtask	Score
Set screw (14)	O
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	X
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	O

Time bonus	-
TOTAL	86

=== PROBLEMS / THOUGHTS

b_bot_outside_camera disconnects. USB has to be plugged back in.
b_bot pulley pick got in the way of a_bot during bearing reorienting --> Need to fix, or make sure pulley is not near a_bot
b_bot did not retighten screws, so we missed one screw for full points Points to improve:
Idler pulley took over 2 minutes, 70 seconds used on insertion alone
Bearing

Done at 11:30 with belt and screws. Idler pulley took until 9:xx. Insertion alone took over 70 seconds (unlucky pick). Motor pulley done at 8:00 Bearing insertion took 25 seconds. Bearing started fastening at around 6:00, finished at 3:30 or so

17.8.2021 (Assembly)

reset 29:20
reset 20:30
reset 18:00

Subtask		Unit 1		Unit 2
G (bearing plate)	2	O	2	O
F (motor plate)	2	O	2	O
B (motor pulley)	3		3
C1 (bearing)	3		3	O
C2 (shaft)	3		3
D (shaft pulley)	3		3
A (motor)	4		4
E (idler)	5		5
H (belt)	5		5
I1 (cable black)	5		5
I2 (cable red)	5		5

TOTAL	40	4	40	7

=== PROBLEMS / THOUGHTS

Reset 1. was due to crash during carry
Reset 2. Fail motor so we skipped
Reset 3. Code problems

17.8.2021 (Assembly)

Reset 25:30
Reset 21:30
Reset 6:30 (after first unit)

Subtask		Unit 1		Unit 2
G (bearing plate)	2	O	2	2
F (motor plate)	2	O	2	1
B (motor pulley)	3	x	3	x
C1 (bearing)	3	O	3	x
C2 (shaft)	3	x	3	x
D (shaft pulley)	3	x	3	x
A (motor)	4	x	4	x
E (idler)	5	x	5	x
H (belt)	5	x	5	x
I1 (cable black)	5	x	5	x
I2 (cable red)	5	x	5	x

TOTAL	40	7	40	3

=== PROBLEMS / THOUGHTS

Motor plate failed, no fallback, realign procedure dropped the tool
We are not prepared well for setting up the state for a reset

9.8.2021 (Assembly)

Reset between

Subtask		Unit 1		Unit 2
G (bearing plate)	2	O	2	O
F (motor plate)	2	O	2	O
B (motor pulley)	3	x	3	x
C1 (bearing)	3	x	3	x
C2 (shaft)	3	x	3	x
D (shaft pulley)	3	x	3	x
A (motor)	4	x	4	x
E (idler)	5	x	5	x
H (belt)	5	x	5	x
I1 (cable black)	5	x	5	x
I2 (cable red)	5	x	5	x

TOTAL	40	4	40	4

=== PROBLEMS / THOUGHTS

Unload got stuck twice, could not complete
No fallback for misaligned L-plates

30.7.2021 (Taskboard)

Subtask	Score
Set screw (14)	O
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	X
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	X

Time bonus	-
TOTAL	72

=== PROBLEMS / THOUGHTS

m3 screw detected as failed
Why do the screw tools activate for half a second after the screw phase?
idler pulley too far in the back left, a_bot failed to pick
idler pulley never enters fallback
after bearing insertion, a_bot motion planning is RRT OMPL --> should be deterministic instead
bearing ROI is not correct, false detections
screw_bearing failed prematurely without any screws fastened. Should be reattempted.

17.8.2021 (Assembly)

Reset 25:30
Reset 21:30
Reset 6:30 (after first unit)

Subtask		Unit 1		Unit 2
G (bearing plate)	2	O	2	2
F (motor plate)	2	O	2	1
B (motor pulley)	3	x	3	x
C1 (bearing)	3	O	3	x
C2 (shaft)	3	x	3	x
D (shaft pulley)	3	x	3	x
A (motor)	4	x	4	x
E (idler)	5	x	5	x
H (belt)	5	x	5	x
I1 (cable black)	5	x	5	x
I2 (cable red)	5	x	5	x

TOTAL	40	7	40	3

=== PROBLEMS / THOUGHTS

Motor plate failed, no fallback, realign procedure dropped the tool
We are not prepared well for setting up the state for a reset

9.8.2021 (Assembly)

Reset between

Subtask		Unit 1		Unit 2
G (bearing plate)	2	O	2	O
F (motor plate)	2	O	2	O
B (motor pulley)	3	x	3	x
C1 (bearing)	3	x	3	x
C2 (shaft)	3	x	3	x
D (shaft pulley)	3	x	3	x
A (motor)	4	x	4	x
E (idler)	5	x	5	x
H (belt)	5	x	5	x
I1 (cable black)	5	x	5	x
I2 (cable red)	5	x	5	x

TOTAL	40	4	40	4

=== PROBLEMS / THOUGHTS

Unload got stuck twice, could not complete
No fallback for misaligned L-plates

04.8.2021 (Assembly)

Reset at 25:00, restart at 23:00
Reset at 18:00, restart at 16:00
Reset at 13:00, restart at 11:00
Reset at 10:00, restart at 8:00
Reset at 5:00, restart at 3:00

Subtask		Success
G (bearing plate)	2	O
F (motor plate)	2	x
B (motor pulley)	3	x
C1 (bearing)	3	x
C2 (shaft)	3	x
D (shaft pulley)	3	x
A (motor)	4	x
E (idler)	5	x
H (belt)	5	x
I1 (cable black)	5	x
I2 (cable red)	5	x

TOTAL	40	2

=== PROBLEMS / THOUGHTS

Reset after protective stop because motor was in the way of the tray (picking tray)
Fail to handover motor plate (missing grasp, last minute change)
Without the plate with can not advance...

30.7.2021 (Taskboard)

Subtask	Score
Set screw (14)	X
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	-
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	-

Time bonus	-
TOTAL	58

=== PROBLEMS / THOUGHTS

28.7.2021 (Assembly)

First run (20 minutes):

Reset at 11:46, restart at 9:26
Reset at 7:40, restart at 5:40

Switched to REAL rules (two products, 30 minutes time):

Subtask		Success (Unit 1)	Success (Unit 2)
G (bearing plate)	2	X	X
F (motor plate)	2	X	X
B (motor pulley)	3	X	X
C1 (bearing)	3	X	X
C2 (shaft)	3	X	X
D (shaft pulley)	3	X	X
A (motor)	4	X	X
E (idler)	5	X	X
H (belt)	5	X	X
I1 (cable black)	5	X	X
I2 (cable red)	5	X	X

TOTAL	40	****	****

=== PROBLEMS / THOUGHTS

Motor plate pick fail undetected
Base plate failed to be picked twice, succeeded in the end
Base plate move_into_center is not set up correctly
Bearing dropped into position that was unrecoverable (diagonal). First retry should be recentering.
lock/unlock should be happening simultaneously at the end of base plate motion
Untested code was in the assembly procedure, causing L-plate failures

21.7.2021 (Assembly)

Reset at 15:00, restart at 12:00
Finished with one screw in the bearing. I gave one point for partial completion.

Subtask		Success
G (bearing plate)	2	O
F (motor plate)	2	O
B (motor pulley)	3	X
C1 (bearing)	3	~
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	5

=== PROBLEMS / THOUGHTS

Reset after protective stop because cable was grasped with the motor plate
Bearing fastening is WAY too slow and unreliable
Unloading failed (due to published collision objects, probably)

16.7.2021 (Taskboard)

Reset at 18:40, restart 15:25
Reset at 12:00, restart 10:00
Stopped at 2:00.

Subtask	Score
Set screw (14)	X
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	-
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	-

Time bonus	-
TOTAL	58

=== PROBLEMS / THOUGHTS

Lockup during screw tasks
M3 screw was missed once but not reattempted
Set screw failed
b_bot took long time to load UR script, process died --> Check load_and_execute_program for robustness
Motors often miss stalling --> Check if this can be detected better
idler pulley grasp/centering pose still a little too high
Idler pulley should be made simultaneous
Robot stopped (goal lost?), had to finish

13.7.2021 (Assembly)

18:30 reset, 15:30 restart

Subtask		Success
G (bearing plate)	2	O
F (motor plate)	2	O
B (motor pulley)	3	X
C1 (bearing)	3	O
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	7

=== PROBLEMS / THOUGHTS

Tray carry failed without protective stop, alignment is incorrect after a_bot touches
Base plate grasp failed, caused reset
Base plate is inclined, should be placed on tray center
Plates could not be grasped with lip by the tray border --> Fix
During plate screwing, suction is sometimes not detected, causing hiccups
Bearing took an extremely long time to be screwed in
Shaft did not initiate fallback procedure
Unloading drive unit caused protective stop --> Release, or adjust drop height (motor, bearing may be in the way)
Tray was not put back on AGC after unloading

9.7.2021 (Taskboard)

Reset at the start
Failure at 9:00

Subtask	Score
Set screw (14)	X
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	-
Pulley (10)	-
Shaft (14)	O
Idler Pulley(14)	-

Time bonus	-
TOTAL	44

=== PROBLEMS / THOUGHTS

M4 screw unequip is very close to getting cable entangled.
Unnecessary b_bot motion to camera view position after carry tray --> Fixed
We should wait for vision more systematically when cameras are dead
Idler pulley grasp position was too high, did not position the pulley correctly so it took more time
MTC pick seems too slow (bearing)
Bearing fastening had an untested change, caused failure --> Fixed
Bearing fastening can still obtain bad IK configurations (!)
Set screw does not go in far enough --> Fixed
Horizontal_screw_ready joint pose should be closer to the actual screwing orientation for the bearing
Infinite loop because of the idler pulley holder --> Improve backup/decluttering procedure

6.7.2021 (Assembly)

With tray carry AND 3D vision
Reset at 17:30, restart 15:15
Reset at 11:15, restart 9:00
reset at 8:15, restart 5:45
Reset 3:30, restart 1:15
Ran out of time before first plate was done (but succeeded after)

Subtask		Success
G (bearing plate)	2	X
F (motor plate)	2	X
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	0

=== PROBLEMS / THOUGHTS

Base plate was accidentally picked together with other plates, caused reset --> Avoid multiple picks
Many resets due to screw function redesign and implementation of vision

2.7.2021 (Taskboard)

Reset at 17:00, restart at 12:45
Protective stop during bearing fastening, finish with 2:30 left

Subtask	Score
Set screw (14)	X
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	-
Bearing (14)	O
Pulley (10)	-
Shaft (14)	-
Idler Pulley(14)	-

Time bonus	-
TOTAL	34

=== PROBLEMS / THOUGHTS

Shaft was picked together with belt --> Prune belt grasp poses better
Belt was not detected in the grasp, dropped on bearing, removed from tray with bearing, failure --> Improve belt detection
IK used flipped joint configuration during fasten_bearing, cabled caused protective stop --> Implement joint flip checks for linear motions
Happened again; why wasn't it caught? --> double check
M4 screw failed for an unknown reason. Maybe the cable got caught? --> Remove edge from the wooden plate to fix the cable situation
M3 screw has an unnecessary movement with a_bot, M4 screw an unnecessary move home
a_bot grasped both shaft and belt, dropped belt outside the tray --> Check gripper opening width after pick
One bearing screw was critically mis-perceived as empty, although screw was successfully fastened. Caused protective stop --> Fix

29.6.2021 (Assembly)

With tray carry, without 3D vision
Reset at 19:10 because the vision workaround is not in the procedures, restart at 17:00
Reset at 10:30, restart at 8:05
Reset at 8:05, restart at 6:00
Drive unit was not unloaded, no second unit started.

Subtask		Success
G (bearing plate)	2	O
F (motor plate)	2	O
B (motor pulley)	3	X
C1 (bearing)	3	O
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	7

=== PROBLEMS / THOUGHTS

Reset was not systematic - current state of assembly/subtasks needs to be tracked
Resets due to gripper colliding with environment during recorded motion --> Fixed
End cap is not found in clutter --> Pick belt and motor before picking small parts

25.6.2021 (Taskboard)

Subtask	Score
Set screw (14)	O
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	-
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	O

Time bonus	-
TOTAL	86

=== PROBLEMS / THOUGHTS

bearing insertion failed
bearing orient down sequence needs fixing
a_bot screwing motion should be optimized
idler pulley nut tool motion should be sped up

22.6.2021 (Assembly)

First time with "full" procedure (tray carry, no 3D vision)
Reset at 19:00, restart at 17:00
Reset at 15:30, restart at 13:30
Reset at 12:30, restart at 10:30
Reset at 9:50, restart at 7:50
Reset at 7:00, restart at 4:30
Reset at 3:40

Subtask		Success
G (bearing plate)	2	X
F (motor plate)	2	X
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	0

=== PROBLEMS / THOUGHTS

Tray carry grasps too deep, can hit objects
Tray carry does not check for protective stop
Tray carry failed because of an IK issue?
Target not listed for base plate
1 Reset because the vision pipeline was not integrated and we did not spawn the base plate correctly
5 resets because tray carry was not stable enough
- 3x lookup bug, 1x protective stop, 1x plan failure(?)

18.6.2021 (Taskboard)

First time with FULL procedure (including tray pick)
Reset at 18:45 because tool unequip failed, restart 16:15
Reset at 10:30 because carry failed
Restart at 4:45

Subtask	Score
Set screw (14)	O
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	-
Pulley (10)	-
Shaft (14)	-
Idler Pulley(14)	-

Time bonus	-
TOTAL	48

=== PROBLEMS / THOUGHTS

Idler pulley failed fatally during pre-testing --> Add error recovery
Belt insertion failed multiple times (a_bot grasp slipped) but was not caught --> Check after motion
Trajectory failed during unequip. This seems to have happened because the robot was already at tool_pick_ready and was told to move there again. This causes an error --> Fix
Unequip procedure failed because of IK failures, caused critical error
Equip set_screw failed because of IK failures
M4 screw failed because robot did not go to the correct position (move to screw pose failed once) --> Retry
Belt should show a_bot camera during pick
Bearing fallback should drop from lower height
Bearing isn't found if it is on its side
Move group died on second bearing pick attempt.
Idler pulley insertion spiral should be faster

15.6.2021 (Assembly)

We did not place or retrieve the tray, or get the drive unit out of the fixation

Subtask		Success
G (bearing plate)	2	O
F (motor plate)	2	O
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	4

=== PROBLEMS / THOUGHTS

New a_bot base plate pick moves the tray --> Fix the tray properly
bearing_plate rearrangement has a blend that with too much jerk
Bearing pick failed and moved the tray backwards
Idler pulley was mistaken for the bearing, protective stop --> Double check with CAD matching after pick
a_bot gripper takes too long to return. Why?
Spiral motion during bearing fastening screw pick does not turn the motor sufficiently long
Screw is sometimes not detected right away after spiral motion --> check

11.6.2021 (Taskboard)

Reset at 17:00, resume at 14:45
Stopped at 4:40 because failure in bearing task caused cascade of bugs

Subtask	Score
Set screw (14)	O
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	-
Pulley (10)	-
Shaft (14)	-
Idler Pulley(14)	O

Time bonus	-
TOTAL	62

=== PROBLEMS / THOUGHTS

idler pulley can enter an infinite loop of being misrecognized as the bearing --> Shifted camera, has to be observed
Tool equip during idler pulley was missed because of IK failure in equip sequence
Unequip motion can fail. We got lucky this time but it should never fail! --> Fix
Bearing sequence failed --> Fix IK
Items being close together can cause failure --> Add grip&drop strategy to avoid clumped items?
Add 2 seconds to motor duration in idler pulley screw
Speed up idler pulley nut tool motion

8.6.2021 (Assembly)

Subtask		Success
G (bearing plate)	2	X
F (motor plate)	2	X
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	0

=== PROBLEMS / THOUGHTS

4.6.2021 (Taskboard)

Finished at 2:50

Subtask	Score
Set screw (14)	0
M3 screw (10)	0
M4 screw (10)	0
Belt (14)	0
Bearing (14)	0
Pulley (10)	0
Shaft (14)	0
Idler Pulley(14)	0

Time bonus	8
TOTAL	108

=== PROBLEMS / THOUGHTS

M3, M4 screws should skip the loosening step
Change active camera during idler pulley
Motor pulley is not detected as successful
Shaft should be done before bearing

1.6.2021 (Assembly)

Skipped (due to ICRA). 0 points because the system was not ready without any preparation.

28.5.2021 (Taskboard)

Reset at 16:00, restart at 11:00
Second reset at 5:30 after dangerous bug --> declared premature finish

Subtask	Score
Set screw (14)	-
M3 screw (10)	O
M4 screw (10)	-
Belt (14)	-
Bearing (14)	-
Pulley (10)	-
Shaft (14)	-
Idler Pulley(14)	-

TOTAL	10

=== PROBLEMS / THOUGHTS

Belt pick accidentally grasped the shaft, but it was returned after the success check. Lovely!
Idler pulley failed because of 45 degree inclination being blocked --> Move the pulley
Set screw procedure reported an error --> Fix the tool collision object
Picking M4 tool was very close to M3 tool
Set screw tool and padless tool were not detached correctly --> Planning failures during screw pick
M4 screw attempted although no screw was picked
Critical Bug: Tools were detached from the robot and caused dangerous plans
LIN motion performance issues. Overall system is too slow, long waits. --> Cartesian controller? Analytical IK?
Idler pulley nut tool placement starts at inefficient position
Unnecessary motion at the start of fasten_bearing

25.5.2021 (Assembly)

The base plate positions were still hard-coded because of the CAD matching problem

Subtask		Success
G (bearing plate)	2	X
F (motor plate)	2	X
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	0

=== PROBLEMS / THOUGHTS

Bearing plate rearranging procedure failed (plate fell off the table) because a_bot pick was too far off. Dangerous.
Motor plate rearranging procedure failed because of the opposite direction once. Was caught after a rearrangement and succeeded finally, but it took a long time.
The new picking with items in the scene is not stable
Tool equipping can fail when too many objects are in the scene (collision object is spawned in an incorrect location). Critical.
Robot performed dangerous motion during bearing fastening (did not find assembled_part_07_screw_hole_1)
Assembly bearing fastening looks very unattractive and risky
Cable gets wrapped around tool during bearing fastening in assembly task

21.5.2021 (Taskboard)

Finished after 13 minutes with no resets (except an issue with the M3 screw feeder at the start)

Subtask	Score
Set screw (14)	X
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	X
Bearing (14)	X
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	O

TOTAL	58

=== PROBLEMS / THOUGHTS

Set screw adapter was too high!
belt was missed (tray was in the way of the grasp pose) but not noticed, so no retry occurred
Idler pulley succeeded but took long
a_bot did not place tool and go back after idler pulley procedure
Bearing insertion was detected as a failure, so it was not screwed in
Bearing was not flush after first screw was attached, screws were not all correctly screwed in --> Fasten all of them one more time at the end
M3 screw feeder did not give new screws after picking

18.5.2021 (Assembly)

The base plate positions were still hard-coded because of the CAD matching problem

Subtask		Success
G (bearing plate)	2	X
F (motor plate)	2	X
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	0

=== PROBLEMS / THOUGHTS

New pick procedure does not move with smooth accelerations --> Tune
Some linear motions seem to have become very slow to plan, might be due to the heavy models in the scene --> Investigate
Tool was not spawned correctly in the gripper once, caused a reset --> Fix issues with large objects in scene
The extreme slowness causes other issues (e.g. tool not attached correctly during equip)

14.5.2021 (Taskboard)

Skipped idler pulley and screw bearing.
Restart from 19:00 to 9:00

Subtask	Score ()
Set screw (14)	X
M3 screw (10)	X
M4 screw (10)	O
Belt (14)	-
Bearing (14)	-
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	-

TOTAL	28

=== PROBLEMS / THOUGHTS

Test was delayed and crashes occurred because of refactoring errors --> Test earlier, avoid big changes
Picking M3 screw failed --> Calibrate systematically ahead of time
Set screw collision object is not correct
Suction is still on after failed screwing --> Fix
M3 failed screwing had a dangerously close motion, does not seem to move back to feeder_pick_ready --> Fix
Motion plans sometimes fail when the robot is already at the target --> Error is spurious, fix this
Belt grasp pose detection failed
It is hard to know what the robot is doing --> Add TV display
Set screw succeeded, but was not screwed in sufficiently! Change the height of the spacer

11.5.2021 (Assembly)

The base plate positions were hard-coded because the CAD matching problem was not solved yet

Subtask		Success
G (bearing plate)	2	(O)
F (motor plate)	2	(O)
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	0(4)

=== PROBLEMS / THOUGHTS

simple_pick first part is slow --> move to above
Bearing plate and bearing failed but we had no retry procedure, so we finished with only the motor plate once

7.5.2021 (Taskboard)

Skipped idler pulley because the tool is not prepared.

Subtask	Score ()
Set screw (14)	O
M3 screw (10)	-
M4 screw (10)	-
Belt (14)	O
Bearing (14)	-
Pulley (10)	O
Shaft (14)	-
Idler Pulley(14)	-

TOTAL	34

=== PROBLEMS / THOUGHTS

M3 screw was missed and not noticed
While M3 screw was being screwed in, b_bot was in home position (should be in a position that pulls less on the cable)
m4 screw was picked but missed (position too high) --> Fix and double check
Pick motion can be much too slow --> Adjust
Bearing angle not found in image, entered long loop --> Find a way to break out or fix the situation
After failing bearing alignment, bearing was searched in tray, but it was not there anymore --> Track progress
It was unclear what the robot was looking for --> Display target on TV
Motor pulley grasp detection failed (too close to border) --> Allow horizontal/vertical grasp
Motor pulley insertion stopped because bearing failed, protective stop --> Don't fail tasks.

30.4.2021 (Taskboard)

Lots of refactoring before this. Skipped idler pulley because the tool is not prepared.

Subtask	Score ()
Set screw (14)	O
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	-
Bearing (14)	-
Pulley (10)	-
Shaft (14)	-
Idler Pulley(14)	-

TOTAL	34

=== PROBLEMS / THOUGHTS

Program stopped because of untested code.
Bug in vision communication caused lockup. Something seems to be wrong with the Action Client. --> Investigate
Screw feeders were not aligned
M3 screw did not finish screwing properly. Height seems to be off.
Belt was missed barely --> Adjust grasping width
Belt tool was not picked correctly...?
UR program loading still failed sometimes
Bearing angle was not detected properly --> Try both with and without LED on
Bearing screw missed and caused serious error --> Make sure bearing screws

27.4.2021 (Assembly)

No vision was used because of a bug

Subtask		Success
G (bearing plate)	2	(O)
F (motor plate)	2	(O)
B (motor pulley)	3	-
C1 (bearing)	3	X
C2 (shaft)	3	-
D (shaft pulley)	3	-
A (motor)	4	-
E (idler)	5	-
H (belt)	5	-
I1 (cable black)	5	-
I2 (cable red)	5	-

TOTAL	40	0

=== PROBLEMS / THOUGHTS

Reset caused because UR program was not loaded in time --> Increased timeout
m4 screw tool got stuck on top of bearing plate --> Print a cone attachment for the screw tool
m4 screw tool missed --> Try moving b_bot gripper further to the right to allow less plate movement
tube got stuck on motor plate --> use 3d print to keep it centered?
cable gets twisted around the m4 tool when screwing in the bearing --> fix motion
Picking the bearing failed because the pose was too close to other objects --> Do not discard grasps as easily
Bearing screws were not fully fastened --> Confirm that position is correct

23.4.2021 (Taskboard)

During pre-testing, a bug during a cartesian motion caused a collision and broke the camera holder. We stopped the test to debug safely and ran a loose follow-up test in the afternoon.

Subtask	Score ()
Set screw (14)	-
M3 screw (10)	O
M4 screw (10)	O
Belt (14)	O
Bearing (14)	X
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	X

TOTAL	44

=== PROBLEMS / THOUGHTS

Untested code was run
a_bot failed to pick screw and was not noticed
b_bot failed to load program, dashboard service does not notice, robot stopped working completely --> Changed timeout settings to fix this
Second look on idler pulley failed often;
TB idler pulley was mistaken for motor and bearing; can cause infinite loop
Grasp poses were discarded because of proximity to items that were already picked --> Clean up memory of items seen in tray

20.4.2021 (Assembly)

No vision used.

Subtask		Success
G (bearing plate)	2	X(O)
F (motor plate)	2	X(O)
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL		0(4)

=== PROBLEMS / THOUGHTS

Screw success with b_bot is not detected correctly, takes too long
First screw got stuck before entering the hole fully --> undetected failure
Linear motion to screw picking is too fast

16.4.2021 (Taskboard)

The first real test with no fixed part positions. Reset at 14:00, restart at 10:15

Subtask	Score
Set screw (14)	X
M3 screw (10)	X
M4 screw (10)	X
Belt (14)	X
Bearing (14)	X
Pulley (10)	X
Shaft (14)	O
Idler Pulley(14)	X

TOTAL	14

=== PROBLEMS / THOUGHTS

Set screw failed twice and was not detected
a_bot screw pick failed both times and was not detected
M4 screwing failed and was not detected
belt pick failed twice and was not detected. Angle was not correct -> get angle from bounding box instead
Idler pulley caused a reset because code was commented out from development --> Do not run untested code
Shaft pick failed once --> Fix grasp angle issue
Bearing was not inserted fully, causing two of the screws to fail. Latter ones succeeded, so this was possible to complete --> Track screw fastening success
Vision result stopped being received by taskboard.py --> Investigate
During testing, belt was picked but failed in the threading because failed motor pulley was in the way --> Change order, aim for more robustness

13.4.2021 (Assembly)

No vision was used because the base plate was missing the PLY file and the small plate could be misperceived (because of symmetry).

Subtask		Success
G (bearing plate)	2	(O)
F (motor plate)	2	(O)
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL	40	0(4)

=== PROBLEMS / THOUGHTS

When fixing the bearing plate, the M4 tool's vacuum connector can get stuck
Reactivating ROS control on a_bot takes a long time after switching to a script
Picking base plate failed because tool was still left in the scene --> Reset properly
After fastening, lifting the tool can lift the plate accidentally --> Set the motor torque to 0 before lifting up

9.4.2021 (Taskboard)

We had changed fingertips and experimented more than usual. Chose to do no restart but to finish the task until the end.

Subtask	Score
Set screw (14)	O
M3 screw (10)	X
M4 screw (10)	X
Belt (14)	O
Bearing (14)	X
Pulley (10)	X
Shaft (14)	X
Idler Pulley(14)	X

TOTAL	28

=== PROBLEMS

a_bot missed picking the screw, M3 task failed
In a previous dry run M3 screw was not fully screwed in --> Add time to screwing
M4 screw missed & failed
Nut for idler pulley was too high, missed
Shaft was grasped 90 degrees rotated, missed
Motor pulley was stuck on the tip of the rod, failed
M4 tool was not in the right position for bearing screwing task, failure --> Add success check and a fallback for screw picking/tool misalignment
Suction is not turned off after fastening --> Fix this

2.4.2021

a_bot was just recently returned from repair. Reset 17:00 to 12:30.

Subtask	Score
Set screw (14)	O
M3 screw (10)	X
M4 screw (10)	O
Belt (14)	O
Bearing (14)	X
Pulley (10)	X
Shaft (14)	X
Idler Pulley(14)	X

TOTAL	38

=== PROBLEMS

The screw tool M3 did not pick the screw correctly. I had no time to recalibrate it correctly. --> Make calibration procedure easier, make screw picking more reliable
M3 screwing was attempted even though no screw was picked --> Add success check
M3 picking pushed the screw out of the way. M4 screws were not coming forward in the feeder --> fix feeders
Shaft was not picked correctly --> Investigate
Shaft was not detected as not picked, because the fingers were in an awkward location --> Add better success check (use camera)
Belt moves things in the tray after picking --> Fix height after pick so it is outside of the tray
Bearing and motor pulley were not executed. Looked like they were not found in the tray. --> Investigate
Idler pulley was in a fixed position, flexible position script did not work yet --> Fix the script

=== THOUGHTS

26.3.2021

a_bot was still out of commission. Restarted immediately at start. Rerun at 16:00

	Real conditions
Set screw (14)	O
M3 screw (10)	-
M4 screw (10)	X
Belt (14)	-
Bearing (14)	X
Pulley (10)	O
Shaft (14)	X
Idler Pulley(14)	-

TOTAL	24

=== PROBLEMS

Forgot commented code.
M4 screw missed, damaged taskboard
Forgot to put shaft into the tray --> skipped
Crash in screw_bearing
Shaft was mistaken for pin and missed --> We need to look closer for parts

=== THOUGHTS

b_bot_inside_camera should be activated at the taskboard start, and then each active screw tool.

19.3.2021

a_bot was still out of commission. Restart from 19:00-17:00. Restart from 14:00-12:00 (because we had time left). Stopped at 10:00 because of a robot connection problem.

	Real conditions
Set screw (14)	X
M3 screw (10)	-
M4 screw (10)	O
Belt (14)	-
Bearing (14)	X
Pulley (10)	O
Shaft (14)	O
Idler Pulley(14)	x

TOTAL	34

=== PROBLEMS

Had to intervene because M4 screw was already in the taskboard from a previous test --> Update checklist
Robot stopped moving because driver node did not notice that ROS_external_control was running on the UR, and stopped accepting goals. Related to the driver version upgrade? Or to a wait_for_UR_program timeout that is too small? --> Investigate
Bearing grasp pose not detected correctly --> Fix
Set screw failed twice --> needs a success check via a_bot camera

=== THOUGHTS

b_bot_inside_camera should be activated at the taskboard start, and then each active screw tool.

12.3.2021

a_bot was missing, so we had many errors due to workarounds for this. 6 resets, not very insightful score. Testing with an incomplete system is a bad idea, but it has to be done.

	Real conditions
Set screw (14)	X
M3 screw (10)	-
M4 screw (10)	X (worked, but was reset)
Belt (14)	-
Bearing (14)	X
Pulley (10)	OK
Shaft (14)	X
Idler Pulley(14)	-

TOTAL	10 (20)

=== PROBLEMS

set screw was missed two times, caused protective stop one time. Added a protective stop release, but no success check! --> Add success check by looking at the back of the taskboard. In real assembly, success can be confirmed via motor turning.
Shaft was attempted to be picked although it was too close to the pulley, causing grasp failure. --> Add visual grasp success check on shaft
Shaft was picked successfully, but failed insertion --> Investigate, add v-groove layover

=== THOUGHTS

Developing until right before the test is a terrible idea. We were planning to put everything under real conditions but did not have time to test beforehand.
We need to move items in the tray with the plunger, or we will get stuck.

5.3.2021

a_bot's gripper did not work, so we used only b_bot. Cameras were used in a simple manner, but only for the motor pulley. Motor pulley was placed with some distance to the other parts, so it was easy to grasp.

	Real conditions
Set screw (14)	OK
M3 screw (10)	-
M4 screw (10)	OK
Belt (14)	-
Bearing (14)	X(missed)
Pulley (10)	OK
Shaft (14)	X(missed)
Idler Pulley(14)	-

TOTAL	34

=== PROBLEMS

a_bot gripper stopped working the night before, so we could only try motions with b_bot
A b_bot unequip M4 tool motion hit the rack again. It looks like the unequip motion was horizontal, which is unexpected -> Investigate!
Shaft and bearing were missed because we are not using the camera feedback -> Fix bearing position estimation
Bearing and shaft motion did not notice that the object was not grasped -> Add success check (grasp width)
Pulley motion was inefficient, it does not understand when it succeeded -> Add position check to stop when insertion has finished

26.2.2021

Reset at 16:00 because the dashboard service did not respond. Restart at 13:30. Stop at 6:00

	Real conditions	Fixed position
Set screw (14)	OK	OK
M3 screw (10)	OK	OK
M4 screw (10)	OK	OK
Belt (14)	-	-
Bearing (14)	-	-
Pulley (10)	-	-
Shaft (14)	-	-
Idler Pulley(14)	-	X

TOTAL	34	34

=== PROBLEMS

Screw feeders did not work correctly (we cheated) --> Wait for second screw feeder to arrive & tune their sensors
Dashboard service crashed --> Socket connection needs to be made more robust
b_bot motor did not detect finished screwing correctly, apparently. --> Check if driver and motor are ok
Idler pulley did not work. Needs a new tool, but we are waiting on the response from the manufacturer
M4 screw tool cable did not retract properly -> Attach spring
Could have continued if the protective stop had been released --> Check more systematically

=== THOUGHTS

19.2.2021

Test was run without cameras (parts positions were fixed, for the 3rd week).
Pretended that M4 screw tool had no suction cup for the idler pulley part (another tool will be ordered for this) Called stop at 13:30 and restarted at 10:00. Idler pulley succeeded the first time around.

	Real conditions	Fixed position
Set screw (14)	X	X
M3 screw (10)	OK	OK
M4 screw (10)	OK	OK
Belt (14)	-	OK
Bearing (14)	-	X
Pulley (10)	-	OK
Shaft (14)	-	X
Idler Pulley(14)	-	X

TOTAL	20	44

=== PROBLEMS

Pre-test:

b_bot seemed to have been interrupted after first 3 screws were done.
error in pulley_v1 program on b_bot upon load --> Solved
belt sequence failed during prep tests. b_bot didn't start. Dashboard must have failed --> Observe (it didn't reoccur)

Test:

Set screw failed (did not enter) --> Add spiral motion?
b_bot screw pick took a long time --> Use tooltip tracking
rviz died, no TV display --> Check
Idler pulley procedure does not stop when screw is fastened, and does not repeat when it fails --> Add recovery!
13:30 stop, b_bot entered protective stop after shaft and did not recover --> Investigate protective stop release function
M4 screw feeder did not refill properly --> Cheated
Missed shaft, but did not stop --> Check grasp success systematically
Bearing was not aligned --> Had to cheat
M4 screw for bearing was not picked, but tried attaching anyway --> add success check
Bearing hole was missed because the tool had rotated in the gripper --> Use tooltip tracking to confirm position
b_bot seemed to keep waiting after motor was done --> Check fastening tool service
b_bot M4 tool unequpping after bearing screwing caused protective stop again --> double check unequip action (does it really apply the extra height??)

=== THOUGHTS

We need to use the cameras to make sure the tools are in the right position! Even just checking via template matching that they did not move would be ok (we can reset the position in the holder)

12.2.2021

The test was done without cameras (parts positions were fixed). Resets called at 19:25, 10:25 and 6:20. Did not attempt another run because taking notes was more important, and we did not want to take risks.

	Real conditions	Fixed position
Set screw (14)	X	X
M3 screw (10)	X	X
M4 screw (10)	X	X
Belt (14)	-	-
Bearing (14)	-	-
Pulley (10)	X	X
Shaft (14)	-	-
Idler Pulley(14)	X	X

TOTAL	0	0

=== PROBLEMS

Forgot commented code, caused reset at 19:25 --> Do not develop just before tests
Idler pulley did not work. Screw tool does not enter. The suction cup seems to be an obstacle --> Tried removing it and it works much better. May need a second tool without the cup.
M3 screw pick took too long, M3 screw tool position seemed off. --> Add check that screw was picked successfully, otherwise equip/center/reequip tool
M4 position was off --> check if screwing was successful, redo with spiral motion if necessary
M4 tool was not unequipped properly, caused restart at 10:25 --> Do not start trials with untested code (no last-minute changes)
At third restart, set screw tool did not move to the correct position. Reason unclear --> Investigate
Set screw tool lost its cover while resetting, blocked restart --> Have replacement ready
Screwing failed more often than usual, can even lift taskboard --> Implement tool-tracking required
A "failed to compile" error message appeared on a_bot during execution. Never seen before. --> Observe

=== THOUGHTS

Last-minute changes are dangerous.
Always use a safe version for scored runs, not an unstable development branch.

5.2.2021

The test was done without cameras (parts positions were fixed). The robot went into protective stop after ~8 minutes. "Finish" was declared at 8:30 minutes because the remaining time was insufficient for another try.

	Real conditions	Fixed position
Set screw (14)	O	O
M3 screw (10)	?	O
M4 screw (10)	O	O
Belt (14)	-	O
Bearing (14)	-	X
Pulley (10)	-	-
Shaft (14)	-	-
Idler Pulley(14)	-	-

TOTAL	24	44

=== PROBLEMS

FATAL: After screwing in the bearing, the M4 screw tool collided with the holder when the robot tried to place it back, causing a protective stop.
CRITICAL: The parts' positions are not obtained from vision!
Screw feeders were not turned on --> Added to checklist
M3 screw feeder does not stop --> We need to buy a new one with working sensors
M4 Screw feeder was slow --> Adjust angle/vibration
Screw feeders were not at the right place (CHEATED) --> Fix position properly
Cable of set screw tool got tangled(!) (CHEATED) --> Add spring to cable
Confirmations were not disabled, so we had to press enter a few times -> "Competition mode" parameter needs to be set to True
There was a bug that kept screwing from being recognized as finished (it took longer than normal, but did not cause issues)
Setting the set screw tool position is finicky; it takes too much time to reload the TB script
Bearing screw hole location was not correct (incorrect location, CHEATED) --> Add procedure to align the holes
Bearing position was incorrect (CHEATED)
Bearing screwing is quite slow
During the belt procedure, the taskboard was lifting off --> Do we need to change the fixation?

=== THOUGHTS

The simultaneous robot motions were very smooth. We will need to use the control on the UR robots more!
We need to make sure that no failures occur after 5 minutes, as a reset becomes infeasible. Trying the difficult things first might make more sense than the current order (setscrew, M3, M4, belt, bearing, motor pulley, shaft, idler pulley). A better order to ensure a reset can be called might be (setscrew, idler pulley, M4, M3, bearing, motor pulley, shaft).

X.X.2021 (Taskboard)

Subtask	Score
Set screw (14)	X
M3 screw (10)	X
M4 screw (10)	X
Belt (14)	X
Bearing (14)	X
Pulley (10)	X
Shaft (14)	X
Idler Pulley(14)	X

TOTAL	0

=== PROBLEMS / THOUGHTS

X.X.2021 (Assembly)

Subtask		Success
G (bearing plate)	2	X
F (motor plate)	2	X
B (motor pulley)	3	X
C1 (bearing)	3	X
C2 (shaft)	3	X
D (shaft pulley)	3	X
A (motor)	4	X
E (idler)	5	X
H (belt)	5	X
I1 (cable black)	5	X
I2 (cable red)	5	X

TOTAL		0