It is incredible how much knowledge and experience of other people is readily available nowadays. Inspired by this fact and also recent advancements in AI, I set out on a quest to accomplish a complex task by imitating behavior and working in a procedural way to determine how an algorithm would approach the problem.
The last day of the year is usually a good time to recall what we have learned, accomplished since January 1st, possibly what we haven’t accomplished and what we are planning for the following year.
Exactly a year ago I challenged myself. Let me share my adventure with you. I hope that it will provide a useful data point and hopefully a source of inspiration for someone.
Just to give you a short introduction. I am a researcher. I like asking questions and figuring out how to answer them. My question was this:
Given the abundance of human knowledge available online, could one learn something non-trivial in a self-taught (unsupervised) manner just by browsing the internet.
While you could quickly say: Yes! or No!, I wanted to do it in a more scientific way. I decided to run an experiment on myself. In this article I will describe what I did and the outcome of this experiment as concretely as possible — because the essence of science is reproducibility, right?
Source: Another World, Delphine Software
I hope that you are still with me. If so, then I’ll say a little bit more about my motivation.
The main reason is the fact that I do research on Artificial Intelligence (AI). More specifically, I am interested in unsupervised learning. This is a kind of AI that could gather data automatically and figure out what to do with it.
There is a nice blog post by Andrej Karpathy about trying to classify images manually in order to determine the human performance on data which would available to AI
I decided that it would be nice to do something like this, but on something more complex.
But that was not the only reason. Many times in my life I have heard that you really need formal education in order to do something and that it is essential to get proper training (by someone who is already a master at something). As a consequence, you need to determine your destiny very early in your life and then live with it. I disagree. I think that it used to be the case, but is not anymore.
Especially in machine learning community there is an ongoing debate on whether it is necessary to possess an official degree or not. I have a PhD degree in Computer Science, so I might be the right person to say that you don’t need one. What you do need is this:
- Curiosity
- Perseverance (this is really important)
- Ability to determine what is true in the data flood
I think I am curious and I can endure a lot, the third one may need some practice, but can be learned. That’s it.
OK. So here is the experiment.
Source: Another World, Delphine Software
THE TASK
Assembly of a 2.5-liter 4-cylinder turbocharged boxer engine (EJ257) with Active Valve Control System (AVCS) manufactured by Fuji Heavy Industries.
EJ25 engine. Source: Subaru
I was lucky to some extent that I had something non-trivial that needed to be done (I had some time I could donate to science). It met all the conditions:
- I really had no idea how to do this. People told me I was doomed to fail.
- There was data (unfiltered) available online
- It looked complex enough that could be used as a proxy problem for something else (I estimated the time needed to accomplish it to be about 100 hours).
- There was a simple YES/NO answer at the end
April 2017, Buttonwillow raceway. I killed this engine then, so I could use it for science. It was dead, so I heard that everything had to be disassembled or replaced.
THE GOAL
Disassemble, repair, assemble. Run for 10000 miles (when you go to a professional mechanic, they will give you this kind of warranty)
THE METHOD
Imitate. Learn by watching a single YouTube video. Don’t do something overly clever.
It worked (passed 10000 mile mark today). Without further ado, I will describe what I did. If you are not interested in details, you can skip to the end.
THE DETAILS
In this section I will:
- Attempt to convince you that the problem is in fact not trivial
- Show the steps taken
Without further ado, I will describe what I did next.
As mentioned before, I had almost zero knowledge about this (full disclaimer: I changed brake pads previously).
This is the video I used and attempted to reverse engineer what was being done. I could assume that the guy (Frank?) knows what to do, because at the end he starts the engine and it works. But all I could do was to watch and imitate. Try it yourself (the is no commentary and everything is sped up)
Based on the YT video, I tried to figure out what are the tools and parts necessary. For that I used Subaru site and ordered parts which I needed (initially I did not even know what I needed, it turned out that the replacement parts cost me $500). I also found various discussions on Subaru boards, but they were not that much useful as there were either conflicting descriptions or people claimed (again), that only mechanics should handle this.
My budget for tools was $1000 (to make it realistic). I made a few purchases of Amazon: Engine Crane (~$300), Impact wrench ($200), Some other wrenches ($200) and hoped I could figure out how to use them.
The experiment officially commenced on January 1st, 2018.
Part I — Disassembly (Jan 1st — Feb 3rd)
The main problem related to this part was to find proper tools and identify all the parts (segmentation?). I also anticipated further issues with figuring out with parts belonged where, so I needed a system to track them.
One pretty useful heuristic when solving this task was to identify all bolts/screws and remove them. I also compared consecutive frames in order to determine what was removed. This was little bit like ‘spot the difference’ game.
LEFT: 3:01 in the video, RIGHT: ten seconds later
From the perspective of imitation learning, I primarily relied on image classification. I actually extracted frames from the video a few times and ran them through reverse image search. The hardest objects were the tools:
What is this?
Piston ring compressor
This one was pretty hard
Can of oil
Pistons
Got this crane of Amazon based on the video
Some more complex examples require action recognition:
This one is easy
Here’s some documentation:
LEFT: Jan 1st, 2018. Impact wrench, crane and coffee, RIGHT: Jan 2nd, 2018. Pulled it out, that’s about 15 hours so far and a approximately 5% of the total work.
LEFT: Jan 20th. Here I already removed most of the stuff, intake and exhaust manifolds are off, wiring is off, what you see is the front — timing belt with 2 cylinder heads on the sides. RIGHT: Jan 24th. Taking off cylinder heads, removing the camshafts.
Jan 25th. Moved inside the house because of rain. LEFT: The block and two cylinder heads. RIGHT: My `workshop`, when I wrote I was doing this by watching YOUTUBE, I meant it! This is the same video linked above.
Feb 1st, LEFT: Pistons are out, RIGHT: The block is split and the only things left are the piston rods and the crankshaft.
Part II — Diagnosis and fix (Feb 4th — March 14th)
Success. I was able to find the root cause of the problem. In order to fix the issue, I needed to replace 4 piston rod bearings. At the same time I decided to get a brand new set of forged pistons and rods, get the crankshaft and cylinder heads machined. Replace timing belt, etc. After all, I was not paying for labor. This part was mostly about figuring out part numbers and waiting.
Feb 4th. I disassembled everything. Found the cause. Can you spot a problem? One of the piston rods was lacking a bearing due to excessive wear.
LEFT: This is a replacement for the part which failed (rod bearing). RIGHT: New pistons.
Part III — Assembly and debug (March 15th — April 17th)
The hardest part (as expected). Here, I needed to purchase a few extra tools:
- A torque wrench ($50)
- Piston ring gap file and ring compressor ($150)
Most of the work was related to figuring out which bolt goes where. The heuristic used here was doing things in the reverse order (I put the parts in plastic bags with time and date during disassembly). There was also a lot of measuring and reassembling things 2 or 3 times, because I missed something.
From the AI standpoint, the hardest part was related to the part at 12:24 in the video.
I had really hard time figuring out what these numbers mean exactly and what they are used for. Eventually, I was able to link the numbers to the manual I got (RIGHT)
SOME more photo-journalism:
These are my notes for doing the assembly. You can see how I planned the order and estimated time in hours. Also, the are some references to the Youtube video (i.e. Oil Cooler 19:20 refers to that point in the video).
Machined cylinder heads, cylinder walls and the crankshaft. ARP head studs mounted.
April 2nd. One of the hardest parts. Took much more time than I anticipated. Adjusting the piston rings required some extra research and math. Pistons expand due to temperature and the gap needs to be just right.
LEFT: Preparing to assemble the most important parts — the rotating assembly — RIGHT
LEFT: Crankshaft is in the half-case, RIGHT: Pistons are in. Only 40h to go…
This part is quite tricky, need to readjust clearances between camshafts and valve buckets before mounting the cylinder heads. 30h left?
LEFT: Block prepared for mounting cylinder heads, RIGHT: Cylinder heads in place. It requires quite a lot of strength to torque them properly.
Putting timing belt back on was tricky too. Almost done. 15 more hours and it should be ready.
Going where it belongs. Just 5 hours left to mount the engine.
Part IV— Evaluation (April 18th — December 31st)
After putting the engine in, there were a few parts which I had to fix (missing wire, etc). This took me about a day, then we everything ‘compiled’, the engine started on April 18th and today I decided to conclude.
During this phase, I tried anything to find an issue, I even used a microscope and looked for something in the oil, but nothing there.
Nerdy me: I used a microscope to look for suspicious stuff in the oil every time I changed it. RIGHT: here’s what you can see if you zoom in a lot. What you should not see: pieces of metal (they would have crystal-like structure).
I have driven 10000 miles (just passed the mark today) and it still works. I have no issues with it. I also checked the power, it is exactly the same as before. Fuel consumption is even lower. Everything is in order.
Conclusion
It took about 100 hours of physical labor and probably twice as much time in research (watching that YouTube video over and over again, probably I watched it about 50 times). I believe that AI is changing the world and we will probably see bots learning from videos like this one.
The hardest part was getting through the noise in information — there are places where you will see conflicting directions. This is where it is kind of useful to have a PhD degree — you are just trained to do this.
Source: Another World, Delphine Software
Overall, I can easily say that now I am at least as qualified as some mechanics I talked with before. Changing oil now takes me about 10 minutes.
The Real Conclusion
If you really want to do something, just do it. The worst is that you’ll learn something and fail, and the best that if you consider what is available online nowadays, it will change your life and give you a lot of satisfaction. Just be prepared to fail a lot and be patient.
Happy New Year! Let this inspire you to learn something new in 2019.
P.S. Let me know if you would like to learn more about this work. I tried not to go too deep.
Future work
Armed with this knowledge, this is obvious, I’m now working on using Reinforcement Learning to swap the original control unit with a Neural Net based one:). Just started working on a piece of code which hopefully can provide more power and lower fuel consumption by learning how to control the combustion process — isn’t anyone working on this?
CREDITS
- The paper ‘Could a Neuroscientist Understand a microprocessor’ (Jonas et al.) provided a lot of inspiration. In order to determine if we are capable of understanding human brain by looking at neurons and activations, the authors wanted to figure out if we can understand a microprocessor using same methods. My question is: `Can a programmer understand an internal combustion engine by watching YouTube?`
Source: Could a Neuroscientist Understand a microprocessor’ (Jonas et al.)