Chipy Mentorship Blog Post #3

Hi everyone! This will be my final blog post as part of the Mentorship program. You can check out my last post here and my GitHub repository here.

As mentioned in my previous blog post, the next big step would be using the data collected from cameras to determine a position from a map. This task can be broken down into multiple different steps:

1. Recognizing key objects on the field (stationary game pieces) and where robot is relative to them
2. Analyzing the direction currently being traveled
3. Updating the current location on a map displaying where the robot is

GUI Implementation:

I began with what I thought would be the easiest task to conquer — task #3. Bin Bin introduced me to tkinter, a python GUI tool that would help visually display this information.

GUI Code

I had a little trouble learning how to use tkinter and apply it to my project, so it took a while to come up with a model. I finally came up with this script, which is supposed to represent the “gps dot” moving across the field. The problem is, I haven’t developed the code to analyze which direction the dot needs to move, so I just have code moving the dot random distances across the map. Once I finish that algorithm, then we can implement that code into the GUI so that the ball moves correctly.

Positioning Based on Key Objects:

This part of the project is far more difficult than others. Not only does the software have to recognize game elements, but also then use that to determine its location. I decided to start with the first goal, which meant that I had to figure out a couple things:

1. Where would I store images of the field elements so that the software has a data set?
2. How would I train a model using the data?

After some research, BinBin and I discovered a couple great options. One was developed by Wyatt Automation and another by Ultralytics. They both use YOLO object detection which makes it easier to implement either one into my existing code. However, I decided that it would be beneficial to use Wyatt Automation’s repository because it allowed me to use a custom data set vs Google’s OpenImagesV4. Currently, I’m making my way through the tutorial. I’ve already uploaded a custom data set, but I’m working on actually training the model so that I can use it effectively. I’ve run into a couple of missing command errors which I’ve been trying to troubleshoot, but its been going pretty well so far.

Moving Forward:

The next problem I’d like to solve is determining the distance between the camera and the field elements from the camera’s point of view. I’ve been researching a little bit about using the scale of objects to determine that distance, but I haven’t found anything concrete that I’m ready to implement yet.

That’s all for now. Since this is my final blog post, I’d like to say that I’ve truly enjoyed being a part of the mentorship program. I’ve learned so many new things that have not only helped further my project, but I know will also be beneficial later in my career.

Once my project is done, I’ll post an update with a video of the code working. Thanks everyone!