Difficulties & Solutions
1. General Path Planning Issues
Since Baxter has a lot of degrees of freedom, inverse kinematics didn’t always compute the best way to get from point A to point B.
Difficulties
We tried to fix this issue by making all of baxter’s movements on the map very small, so that there would be a lower chance that MoveIt would return a crazy path for the arm to move from point A to point B. However, this cause numerous issues. Baxter did not do well moving extremely small and precise distances. Often it would start shaking, or end up not moving at all. Other times, the whole process was way too slow, so we decided to scrap this idea to fix the path planning issue.
Solutions
Again more research! Path planning is a hard problem to solve, and maybe MoveIt isn’t the best way to do IK. We could have perhaps written our own IK function to calibrate for errors for this project specifically, but that would have taken time we didn’t have.
2. AR Tags
AR Track Alvar was very finicky, and caused numerous problems with map construction and origin position locating.
Difficulties
Solutions
AR Track Alvar was very finicky, and caused numerous problems with map construction and origin position locating.
Difficulties
- Origin detection: In order to build our map with relative coordinates, we had to maintain an origin throughout the entire process. Without reference to that origin, the robot would not know where its arm was in space, and it would cause our whole model to collapse. So, in order to maintain a constant reference to this origin, we used Baxter’s right hand camera to detect ar_marker_0, our origin, at all times. A challenge we faced however, is that the right hand camera kept losing track of the marker, and because the origin of our map would be lost, our robot would start moving in crazy directions because it didn’t know where its arm was in space.
This is a picture of the right hand camera view detecting AR marker 0. As we can see here, the quality was extremely poor, and detection caused us a lot of issues. - Two-Step Process: We originally wanted the process to be completed in one step. We wanted to use the head camera to detect the AR tags and build the map, while maintaining constant reference to the origin, and immediately begin execution of the algorithm afterwards; however, the AR tags were too small for the head camera to detect. So, we were forced to split the process into two steps
- Use the USB camera to detect the AR tags and build the map
- Once the map was built, flip the blocks over to hide the AR tags, and run the algorithm, making sure consistently that ar_marker_0 was in view at all times.
Solutions
- Eliminate AR Tags completely. Color the blocks differently based on block type, and use color tracking to detect the different kinds of blocks. In the same way, we can draw patterns on each block based on block type, and use pattern matching to detect the different kinds of blocks in the game. OpenCV and other image processing libraries have this sort of capability we can code into Baxter, and with a little more research we believe it would be much more accurate. This would also remove the step where we have to flip the blocks to hide the AR tags, and bring us closer to a one step process.
- To bring us even closer to a one step process, we could have done one of two things. Since we knew that we couldn’t use the head camera to detect the AR tags, and we knew that the right hand camera of Baxter was extremely bad, we could have tried our luck at the left hand camera keeping track of the origin of the map. As it turns out, the quality of Baxter’s left hand camera is much better than the quality of its right hand camera. So perhaps an easy fix would be if we use baxter’s left hand camera to keep track of the origin, as well as use it to build the map! Another thing we could have done is attach the USB cam to the chest of Baxter in a way that it could capture all of the block locations.
3. More complicated Maps
Currently, we only have 2 boxes and 2 destinations, so the maximum # moves on the map is 3.
Difficulties
We tried to extend this to more than 2 boxes but we faced one model constraint and one logic constraint.
Solutions
More research! Efficiency was the issue, so if we could just find a way for the stack of moves to be wider and not deeper, we’d have a solid solution!
Currently, we only have 2 boxes and 2 destinations, so the maximum # moves on the map is 3.
- Case 1: 2 Moves:
- 1st Move: Box 1 to Destination 1
- 2nd Move: Box 2 to Destination 2
- Case 2: 3 Moves:
- 1st Move: Box 1 moved aside to clear a path for Box 2 to Destination 2.
- 2nd Move: Box 2 to Destination 2
- 3rd Move: Box 1 to Destination 1
Difficulties
We tried to extend this to more than 2 boxes but we faced one model constraint and one logic constraint.
- Model Constraint: Baxter’s reachable workspace couldn’t allow for a larger map than one that could only use 2 boxes. We could have made the boxes and our Sokoban grid smaller, but then the AR tags on the boxes would be too small to detect, so we had to stick with two boxes only .
- Logic Constraint: The stack of moves that would have to be executed by baxter for more than two boxes gets awfully deep, and we ran into a lot of memory/segmentation faults when we tried to extend the case to three + boxes. We tried to make a simple logic extension on our code for 2 boxes, but we would have had to sculpt our logic entirely differently to make our logic extendable.
Solutions
More research! Efficiency was the issue, so if we could just find a way for the stack of moves to be wider and not deeper, we’d have a solid solution!
4. Pushing vs. Grabbing the Box
If we figured out a way to push the box instead of lift the box, we would model the original Sokoban game better, because in the original game the boxes are pushed! An example of our model of the game being different due to the fact that we grab boxes and not push them, is best seen in the Corner Example. The Corner Example is the case where one of the boxes is in an L shaped corner, blocked by walls on two sides. In the original Sokoban game, there is no way to push the box out of this configuration, and the game is over. In our Sokoban game however, there is a way out of this, because we can grab the box from the top and move it if it is in this scenario.
Difficulties
We tried to implement pushing, but it became too difficult with the friction of the block and the map. Sometimes, pushing the block would cause the map to get wrinkles, being detrimental to our map building, because distances between boxes would not be calculated correctly due to the wrinkles in the map that our usb camera would pick up.
Solutions
We could have gotten a wooden surface that doesn’t wrinkle, and oiled up the surface and the blocks so that pushing went smoothly. That would just be a budget thing, but we think would solve the pushing problem entirely because we’d be minimizing friction.
If we figured out a way to push the box instead of lift the box, we would model the original Sokoban game better, because in the original game the boxes are pushed! An example of our model of the game being different due to the fact that we grab boxes and not push them, is best seen in the Corner Example. The Corner Example is the case where one of the boxes is in an L shaped corner, blocked by walls on two sides. In the original Sokoban game, there is no way to push the box out of this configuration, and the game is over. In our Sokoban game however, there is a way out of this, because we can grab the box from the top and move it if it is in this scenario.
Difficulties
We tried to implement pushing, but it became too difficult with the friction of the block and the map. Sometimes, pushing the block would cause the map to get wrinkles, being detrimental to our map building, because distances between boxes would not be calculated correctly due to the wrinkles in the map that our usb camera would pick up.
Solutions
We could have gotten a wooden surface that doesn’t wrinkle, and oiled up the surface and the blocks so that pushing went smoothly. That would just be a budget thing, but we think would solve the pushing problem entirely because we’d be minimizing friction.
Discussion
Our whole group agreed that, for a rudimentary game of Sokoban, our robot and algorithm performed very well. There were no unexpected bugs, and for the gripper model of the game we had imposed, the robot performed exactly how it was supposed to most of the times (the times it didn’t perform as expected are discussed above in the “AR Tags” Section and the “General Path Planning Issues” section).
We had about a 90% accuracy for the simple maps we constructed requiring 2 and 3 moves. The 10% of times it didn’t work was due to a buggy AR tag or horrible path planning by MoveIt.
As discussed in the solutions above , we think if we had more time, we’d be able to make a robot that plays the game well enough to put it into production!
We had about a 90% accuracy for the simple maps we constructed requiring 2 and 3 moves. The 10% of times it didn’t work was due to a buggy AR tag or horrible path planning by MoveIt.
As discussed in the solutions above , we think if we had more time, we’d be able to make a robot that plays the game well enough to put it into production!