Launch HN: Flywheel (YC S25) – Waymo for Excavators
Hey HN, We're Jash and Mahimana, cofounders of Flywheel AI (https://useflywheel.ai). We’re building a remote teleop and autonomous stack for excavators.
Here's a video: https://www.youtube.com/watch?v=zCNmNm3lQGk.
Interfacing with existing excavators for enabling remote teleop (or autonomy) is hard. Unlike cars which use drive-by-wire technology, most of the millions of excavators are fully hydraulic machines. The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints. This means excavators mostly do not have an electronic component to control the joints. We solve this by mechanically actuating the joysticks and pedals inside the excavators.
We do this with retrofits which work on any excavator model/make, enabling us to augment existing machines. By enabling remote teleoperation, we are able to increase site safety, productivity and also cost efficiency.
Teleoperation by the operators enables us to prepare training data for autonomy. In robotics, training data comprises observation and action. While images and videos are abundant on the internet, egocentric (PoV) observation and action data is extremely scarce, and it is this scarcity that is holding back scaling robot learning policies.
Flywheel solves this by preparing the training data coming from our remote teleop-enabled excavators which we have already deployed. And we do this with very minimal hardware setup and resources.
During our time in YC, we did 25-30 iterations of sensor stack and placement permutations/combinations, and model hyperparams variations. We called this “evolution of the physical form of our retrofit”. Eventually, we landed on our current evolution and have successfully been able to train some levels of autonomy with only a few hours of training data.
The big takeaway was how much more important data is than optimizing hyperparams of the model. So today, we’re open sourcing 100hrs of excavator dataset that we collected using Flywheel systems on real construction sites. This is in partnership with Frodobots.ai.
Dataset: https://huggingface.co/datasets/FlywheelAI/excavator-dataset
Machine/retrofit details:
Volvo EC380 (38 ton excavator)
4xcamera (25fps)
25 hz expert operator’s action data
We’re just getting started. We have good amounts of variations in daylight, weather, tasks, and would be adding more hours of data and also converting to lerobot format soon. We’re doing this so people like you and me can try out training models on real world data which is very, very hard to get.
So please checkout the dataset here and feel free to download and use however you like. We would love for people to do things with it! I’ll be around in the thread and look forward to comments and feedback from the community!
> The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints
I've actually spent a decent amount of time running an excavator, as my Dad owns a construction / road building company. It was a great summer job!
An important note about the pilot hydraulics is that they _provide feedback to the operator_. I would encourage any system that moves these controls on behalf of a remote human operator or AI to add strain gauges or some other way to measure this force feedback so that this data isn't lost.
The handful of "drive by wire" pieces of equipment that my Dad or other skilled operators in my family have ran were universally panned, because the operators are isolated from this feedback and have a harder time telling when the machine is struggling or when their inputs are not sufficiently smooth. In the automotive world, skilled drivers have similar complaints about fully electronic steering or braking systems, as opposed to traditional vacuum or hydraulic boosting approaches where your foot still has a direct hydraulic connection to the brake pads.
You're right! This is exactly why we like to do mechanical actuation - we are able to achieve bilateral telepresence, which essentially gives the torque (haptic) feedback over the internet! So on small excavators, you can absolutely feel the resistance. We also stream the engine audio, which tells you how hard the hydraulic pump is working. Operators like our system for these reasons :)
I'd like to get a chance to talk to you and your Dad to get feedback. How do I reach you? My email is contact at useflywheel dot ai
Not my industry at all.
I am curious if something like this is an opportunity for a whole new type of controls and feedback. Since the operator doesn’t have to be in the excavator physically they could take on any position: standing, sitting, lying down, etc. Instead of sending haptic feedback to the joystick it could be sent to a vibrating wrist band. You could hook up the equivalent of a Nintendo Power Glove to have the scoop operated by the operator simulating scooping action. Turning the excavator can be controlled by the operator turning their head and moving it around can be done by the operator walking on an infinite treadmill. Motor strain can be done via color of light or temperature rather than sound. You could have a VR helmet that can also show you a birds eye view from a companion drone, overlay blueprints, show power and water lines, measure depth, etc. I don’t know if it is possible but maybe you could even measure soil composition somehow to show large rocks, soft soil that is dangerous to drive over, inclination angles where the excavator is about to drive, etc.
I imagine skilled operators prefer familiar controls but perhaps there are interesting improvements unlocked by exploring alternatives. It might also fundamentally change how accessible it is for non-professionals to use these machines. I rented an excavator from Home Depot a few years ago to dig a foundation and the learning curve was not shallow. I wonder if a more “natural” interface would help keep people safer.
These are really really interesting thoughts around the remote teleop interface. A few things you mentioned have been on my mind. Teleop UX is underdeveloped and its affects is underestimated, and it could turn out to be a huge thing for us and humanoid companies if autonomy is harder than it seems now. I don't believe construction equipment operation today is optimal. We just go with it as that's what we have. It's far from intuitive and easy for a newbie to start operating to give some context and requires hours of practice. There would be a bit of training to be done for remote teleop as well. Might as well make it easy to interface and improve on the existing experience, which hasn't really evolved for years and decades!
I'd like to have a chat with you if you're up for it: contact at useflywheel dot ai
I have a 20 ton Takeuchi and I don’t recall any feedback in the controls at all. The feedback I use is from the seat and sounds of the machine - well besides of course visual of course.
I cannot imagine this being useful to me unless the virtual operators cab closely mimicked an actual machine. It would have to have audio from the machine and be on a platform that tilted relative to the real thing. It would also need 270 degrees of monitors with a virtual mirror to see behind. On the front monitor, minimally, would need the to see vertically up and down too.
I also imagine all of this would be more useful to seasoned operators who can do most things on excavators in their sleep (definitely not me lol)
The way I think about this - we should not have multi screens. Human field of vision is 60 degrees for central and about 120 degrees binocular. The bucket of the excavator is way narrower than this which means actual task doesn't require wide vision.
So if we are able to have really good autonomous safety layers to ensure safe movements, and dynamically resize remote teleop windows, you'd make the operator more efficient. So while we stream 360 degree view, we get creative in how we show it.
That's on the vision side. We also stream engine audio, and do haptic feedback.
Takeuchi are interesting! Rare ones to have blades even on bigger sizes - is that why you got one?
Just a suggestion from someone who's worked on industrial robots and autonomous vehicles, but I think you're underselling a lot of difficulties here.
Skilled humans have a tendency to fully engage all of their senses during a task. For example, human drivers use their entire field of vision at night even though headlights only illuminate tiny portions of their FoV. I've never operated an excavator, but I would be very surprised if skilled operators only used the portion of their vision immediately around the bucket and not the rest of it for situational awareness.
That said, UI design is a tradeoff. There's a paper that has a nice list of teleoperation design principles [0], which does talk about single windows as a positive. On the other hand, a common principle in older aviation HCI literature is the idea that nothing about the system UI should surprise the human. It's hard to maintain a good idea of the system state when you have it resizing windows automatically.
The hardest thing is going to be making really good autonomous safety layers. It's the most difficult part of building a fully autonomous system anyway. The main advantage of teleop is that you can [supposedly] sidestep having one.
[0] https://doi.org/10.1109/THMS.2014.2371048
I definitely agree with you - recreating the scene in teleop is challenging. In excavators, however, it does make it better. An excavator has huge blindspots on the right (due to arm), to the back and sometime near the bucket. Hence the workers who are hired to stand around (banksmen, spotters, signalmen) and signal to the operator.
It's like driving a Ford F150 without backup camera. You'd add the backup camera upfront, and not display the back view at the back window.
It's definitely challenging and we're far from something that's perfect. We're iterating towards something that's better everyday.
Yeah, it sounds like a fun challenge. Hope you have lots of success tackling it
Well sure if you are just looking at where the bucket is digging but there is often a dump truck sitting on either your right or left flank waiting for what’s in your bucket (don’t forget the beep button lol). Having a monitor to either side duplicates what you are seeing out of your peripheral vision when operating the real thing. Would make transitioning from real to virtual much easier and imho safer.
Yes that is precisely why - makes for a much more versatile machine. TB180FR - it’s med-small, about 10 ton.
I think swinging (which is about 40% of dig and dump workflow by time spent) should not be manual. That's lowest levels of autonomy which requires roughly centering to the pit/truck which we have already achieved. Hence operator only has to look in front!
Those workflow numbers come from multiple observations at different sites, one of the examples is this: https://www.youtube.com/watch?v=orEsvu1CS64
I wish to talk to you because it's rare sight someone has a Takeuchi - is there a way to connect? My email is contact at useflywheel dot ai
>>I think swinging should not be manual.
I disagree and here's a couple of reasons why I say that:
1. What am I going to do with the time between releasing control and regaining it from the autonomous control?
2. In that break of workflow my first thought is it will cause a break in my concentration.
3. When I am swinging back from the truck to the trench the bucket is naturally in my control. It seems that in autonomy mode the transition from autonomous to my control would be very unnatural and choppy. I suppose with time it would be okay but man seems to violate the whole smooth is fast concept.
I'll shoot you an email.
> I think swinging (which is about 40% of dig and dump workflow by time spent) should not be manual.
It's been over a decade since I last operated an excavator, so grains of salt as usual - but I'd say it should be manual, or at least semi-automated. You need to take care where you unload the bucket on a truck, to avoid its weight distribution being off-center, or to keep various kinds of soil separated on the bed (e.g. we'd load the front with topsoil and fill the rear with the gravel or whatever else was below.
I agree - the dumping and digging itself (where you move boom, arm, bucket much more than swing/tracks) should be manual. But swinging to the truck and back to pit (pure swinging motion to center around these areas of interest) do not have to be manual. I agree with your and other comments that the transition has to be smooth and that's something we're working on.
Yes, and you also get feedback from your butt as the machine tips and wobbles, particularly on smaller machines. Hearing the engine straining helps also. Often you can not clearly see what you are digging, this feedback lets you know if you are running into a rock or something.
One big advantage would be cameras mounted on the boom and rear view cameras, as many machines have obstructed views.
We're indeed streaming the audio and have haptic feedback. My hypothesis is the seat vibration isn't as helpful. It's sub optimal and operators would be far more productive without it. We would do a paper when we have enough data on this. We're also putting more cameras but streaming a lot of them at once is tricky.
Humbly, have you used excavators of varying sizes on uneven ground? I have and would suggest it's more important than you might think. But if you've operated them, you might know better than I.
Also, teleoperation is likely to produce lower-quality operation data than hooking up to locally operated excavators. Just a thought.
I might be less experienced than you - I've operated upto 38 tons on maximum 15 degrees incline. I wasn't moving tracks a lot when I did that. Would like to hear what scenarios you'be been in and how would you describe your experience? Maybe I could try those out to learn more!
The problem is how do you quantify it and metricize it and get some numbers to compare? Because it's absolutely invaluable after you get to a level of mastery with a machine that you're sitting in that the seat moves in. You just feel it, you can't explain it in words, you're just one with the machine. Your butt really needs to feel that in order for that to really happen. Can you do it without it? Absolutely. Does it make it worse to not have that? Also true. It's extra effort, it's extra cost, and it makes things better for the actual users of the product, but it'll never show up as something you can measure.
Do you want to sell Microsoft Teams to the executives or do you want to give joy to the people who actually use the product?
You've touched one of my favorite things to think about. I think about this a lot. There will always be people who enjoy fully analog cars/planes/watches over alternatives. I think that doesn't define the whole market. How many of new joinees are we seeing in this industry? Would you rather have a functioning back because you operate these remotely or go to site and sit in one of these and have a broken back by 50s? This is leaving out the fact that remote operation could save lots of lives. In our launch video we did put together some of the news clips on it: https://www.ycombinator.com/launches/O8i-flywheel-ai-waymo-f...
My car with its drive-by-wire brakes has a brake feedback simulator that gives the driver the kind of feeling associated with power-boosted hydraulic brakes. This is by far the most expensive single component in the car. Arguably these are just expensive accommodations for human flaws. A self-driving car wouldn't need them. Can't the self-driving system act directly on data like pressure, flow, and displacement?
That's indeed what we're trying to test to the extreme - to see how far we could go with just vision. We haven't done heavy excavation workflows yet, but we have some early success with some excavation workflows with just vision input and joystick action output (even without joint angle feedback!). We're betting on having really huge data with compact observation input and experiment to see if it holds water. If not we can always dial it down and add more sensors/feedback.
Maybe it doesn't matter for a car because feeling the car's motion tells you most of what you need to know. A car is not meant to touch anything but the road, in normal conditions. I think steering is the only case where force feedback is very important for a car - In the winters up here, I can feel the steering go loose when I hit a patch of ice.
I imagine an excavator, meant to touch and dig through things, and lift things, benefits from force feedback for the same reason VR would.
Have you played those VR sword games? BeatSaber works great because you're cutting through abstract blobs that offer no resistance. But the medieval sword-slashing games feel weird because your sword can't impact your opponent.
I saw a video recently of a quadcopter lifting heavy objects. When it's overloaded, it can't maneuver because all its spare power is spent generating lift to maintain altitude. If the controls had force feedback, the copter's computer could tell you "I'm overloaded, I can't move" by putting maximum resistance on the sticks.
Interestingly, we had some people try out VR teleop: https://x.com/Scobleizer/status/1970245161306464667
https://x.com/jash_mota/status/1969091992140304703
I think force feedback is key for small excavators, but not really true for 25+ tons excavators. Hence how easy it is for operators to accidentally kill someone with it.
There are no drive-by-wire brakes in the US or Europe for regular cars. Your car's actuator moves the piston that is mechanically linked to your pedal.
So even if the electric system fails completely, you can still actuate the brakes.
Toyota's hybrids, at least, have valves in the hydraulic system. If everything is working, the driver's pedal is isolated from the physical pistons. Pressing the pedal instead moves a 'stroke simulator' (a cylinder with a spring in it), and the pressure is measured with a transducer. The Brake ECU tries to satisfy as much braking demand through regenerative braking as possible, applying the rear brakes to keep balance and front brakes if you brake too hard, requesting more braking than can be generated or the battery can absorb.
If there's a failure of the electrical supply to the brake ECU, or another fault condition occurs, various valves then revert to their normally-open or normally-closed positions to allow hydraulic pressure from the pedal through to the brake cylinders, and isolate the stroke simulator.
Because the engine isn't constantly running and providing a vacuum that can be used to assist with brake force, the system also includes a 'brake accumulator' and pump to boost the brake pressure.
Reference: https://pmmonline.co.uk/technical/blue-prints-insight-into-t...
I don't know for certain, but I would assume that other hybrids and EVs have similar systems to maximise regenerative braking.
It depends on how you define brake by wire, but the one I'm referencing is the C8 Corvette and it's "eBoost" system. This isn't a purely electronic system like throttle by wire is, but it does mean there is no longer a linear relationship between pedal pressure and brake pad pressure. And my point about isolating the driver from feedback still holds true.
[flagged]
Would you be able to replicate this with the heavy equipment and movements needed to plug orphaned oil wells? In Texas alone, it's a TAM of ~$38B, and ~$150B for the entire US.
The Looming Disaster Under America's Biggest Oil Field [video] - https://news.ycombinator.com/item?id=45361022 - September 2025
Texas has thousands of abandoned oil and gas wells. Who is responsible for cleaning them up? - https://www.texastribune.org/2025/05/08/texas-orphan-wells-e... - May 8th, 2025
The Rising Cost of the Oil Industry’s Slow Death - https://www.propublica.org/article/the-rising-cost-of-the-oi... - February 22nd, 2024
Well plugging SOP:
https://www.epa.gov/natural-gas-star-program/well-plugging
https://www.osha.gov/etools/oil-and-gas/abandoning-well
It is possible. One good thing about our retrofit is it works on any hydraulic equipment, and not just excavators, although we are focussed on excavators for now. We are able to retrofit any hydraulic machinery because we actuate the joystics connected to the valves, which are quite standardized and usually come from handful of companies like Bosch. Will definitely 'dig' more into this though.
Your service will need to be cheaper than just lobbying the current federal government.
Skate to where the puck is going. By the time they're ready for prod (2-4 years), regime change might have occurred. Be ready with a solution to this environmental problem at that time. Lots of oil and gas infra over the next few decades that will require remediation after the world transitions to low carbon energy and electrification, while declining fertility rates globally will mean a shrinking pool of workers to pick from for the human labor component.
Locales who don't want their aquifers contaminated will also be motivated in the near term to get this work done. They would be great test cases for proof of value of this automation imho.
(think in systems)
Very optimistic thinking in your first 2 sentences there...
Says somebody who has clearly never worked in oil and gas.
I haven't worked in the industry, but a family member does and owns several wells (ie the mineral rights) in Texas, passed down from their deceased parent who was a rig worker.
If you would be so kind as to suggest resources so that I can better educate myself in this domain for any missing nuance, I am open to any and all resources, including me traveling on my own dime to speak with a subject matter expert in person (who I am willing to compensate at a reasonable hourly rate).
Talk to people working in oil and gas at the executive level, or even better, try selling stuff to them. Oil and gas operates on a very different mindset than the nibbling-at-margins product-led-growth models that are so prevalent on HN.
The only metric that matters at the end of the day is how to print more money i.e. drill more oil. Literally nothing else matters, and compliance goes to the lowest bidder to make the problem go away. Right now that means playing nice with the whitehouse and hopefully bypassing the rest of the pesky EPA. I assure you not a single person in oil and gas likes the EPA, nor the infamous Subpart W.
Could a mini tunnel boring machine plug a well, from the side?
That's definitely a thing.
https://en.wikipedia.org/wiki/Relief_well
Is there a name for resealing an aquifer at each layer? Rezonal isolation, Zonal isolation, Zonal Re-isolation?
/? Zonal isolation Wikipedia: https://www.google.com/search?q=zonal+isolation+Wikipedia :
> Well cementing, Completion, Squeeze job, Cement bond log, Casing (should prevent fluid from contaminating e.g. aquifer zones)
Orphan wells: https://en.wikipedia.org/wiki/Orphan_wells :
> they estimate there are 29 million abandoned wells internationally
Orphaned wells in the United States: https://en.wikipedia.org/wiki/Orphaned_wells_in_the_United_S... :
> According to the Government Accountability Office, the 2.1 million unplugged abandoned wells in the United States could cost as much as $300 billion
> TAM of ~$38B, and ~$150B for the entire US.
Sigh. No. Actual TAM is $0 which is why these wells are orphaned
For contrast, maritime shipping IMO (my current obsession) has set a $100-$400 carbon price. So now the efuel market a realistic $10B ZNZ pot to aim at.
What a fantastic business. I've watched lots of digger (as we call them here) drivers in action and I'm in awe of what these machines can do and how incredibly skilled their operators are. We have several large rock walls on our property and watching these guys delicately picking and placing huge rocks to key them in just blows my mind.
There was a YouTube video recently of an AI-assisted digger making a wall out of the concrete rubble from a demolition.
I believe the applications for really smart excavators must be huge. Sounds like this might be a step on that voyage.
You're spot on! The experienced operators are quite skilled at moving these machines. The number of operators who could do this work are decreasing and it's also pretty dangerous which explains the reduction of new people joining in, especially with better ways to make money today. We're getting started with remote teleop and moving towards full autonomy!
Robot Excavator Builds Massive Stone Wall Without Human Assistance: https://www.youtube.com/watch?v=Belu_1pUPVk
This looks very interesting. I understand that hydraulic retrofit is compelling, but you may want to consider CANbus-based retrofits as well (similar to how CommaAI works, where it simply sends messages to the CANbus network to control steering or speed just like ADAS features do). Modern equipment is moving away from hydraulic controls toward a fully digital cockpit and being able to plug in and teleop (for less than the cost of the manufacturers expensive robotics suite) would be amazing.
I have a Cat 289D skid steer I would happily contribute to the effort if you guys move into the compact equipment space (compact being a relative term, as my machine is only 6 tons compared to your 38 ton machine)
Thanks! Yes we're testing both CAN spoofing and retrofits. CAN spoofing really limits compatibility to a few machines/brands. Would love to get in touch with you: my email is contact ai useflywheel dot ai
I disagree - your AI model should be decoupled from your control logic; so if the model determines that you need to move the boom, your control logic takes that action - either via hydraulic retrofit, canbus control, or just faking it in a simulator, and the AI model observes and reacts to the change as a result of that action.
So in reality, CANbus control should be no different than hydraulic control to your system - just another style of I/O
I totally agree, what I meant was if we only do CANbus retrofit, we would really limit ourselves in how compatible we are. So we have to focus on doing the right kind of retrofit really well. CANbus retrofit is easier and we've done that many times before. Especially because my cofounder Mahimana used to work in self-driving company and worked on this and single-handedly built out their current ML data and eval pipeline.
I would create a bespoke training set from the very best operator you can find, after significant interviewing, and then only use that to train your model (eg. partner with a local firm that rents at Herc or United Rentals a lot and has a skilled operator and train on their jobs while you rent them free equipment).
The range of skillset on these things is large, there are 2 dominant (fairly swappable - meaning most machines do both) different operating modes (excavator / backhoe) and I see that operators have a specific one they are best with.
Honestly, when I see a real pro using one of these machines I think this is one area AI is not going to win at soon - in the real world there's a whole support crew working with the excavator operator
This is a great idea - I would dig into this. I disagree on AI not winning here. Support crew mostly work on eliminating blindspots for operator. Yes you can eliminate blindspots with cameras and a 7inch TFT screen, but those are small, don't work great in direct sunlight. The current solution not being 10x better is why they still fallback to having those crew members. I genuinely believe we will change that.
Would be really cool if you could set up a pilot on https://www.youtube.com/@dustupstexas
I would love to explore that - could you help make this happen (I'm guessing you know him)?
Whats the budget for getting an MVP for heavy equipment ?
I ask in all seriousness since, for example, retrofitting regular semis to electric requires millions and millions just to get started
I feel like the inherent bootstrap cost of hardware startups is usually reflected in fundraising amounts.
It's far cheaper than millions! We're making something equipment owners want :) Let's chat? My email is contact at useflywheel dot ai
> I ask in all seriousness since, for example, retrofitting regular semis to electric requires millions and millions just to get started
That's because semis - just like cars - are road legal and everything has to be done by the books. The hard part isn't swapping out the engine and mounting a battery pack, car modders have done that for decades for funsies, the hard part is getting it certified for roadworthiness.
On construction sites, particularly ones fully on private property, no one gives a fuck about the equipment, at least not until someone gets hurt, maimed or killed.
I'd like to add that this was done in partnership with Frodobots.ai (https://frodobots.ai) and BitRobot.ai (https://bitrobot.ai). They are working on a great mission statement and were amazing partners!
What do you think of Komatsu's excavator teleoperation system[1], and what are you doing differently than they are?
[1] https://youtu.be/QKMsqFMcmL0&t=962
1. We're OEM/machine-agnostic 2. Full compatibility with any existing model and not just specific recent models
This scratches the itch I've had since playing with Tonka trucks in the back yard when I was 5... Are you hiring?
Thanks! Let's chat: contact @ useflywheel dot ai
> Flywheel
Hails cab. Excavator shows up
Haha yes - luckily we don't get this outside SF. We call it flywheel because our mission is to solve data flywheel problem in robotics.
related, this cool photo: https://x.com/jash_mota/status/1969635994220839095
I have a question about competition. I had a friend who was in mining about a decade ago. He told me that their mines were fully autonomous back then -- that the equipment managed itself and sent back sensor data.
Is that different than what you are doing now?
Mining has had autonomous haulage (and to some extent excavation) for some years. They have been very tightly coupled to the individual machines, models and the quarry site.
Construction excavation workflows tend to be different and more varied (hence you would see more variations in the machines/attachments available for this industry). And so far there's no solution which just works or is plug and play. This is an attempt to solve that.
And we believe this is great time to do so due to market demands too. No way Gen Z is going to do these jobs when you can make money being delivery partner/influencer, etc. The high-certified, high-skilled workforce is going to shockingly shrink in some years.
How much safety training have you done with the models? i.e. does it know to stop if it's about to drive over a human?
We have some autonomous safety layers (comprised of object/human detection like you mentioned and others) even for remote teleop. But we haven't deployed autonomy on real sites yet. We still have some work to do before we can reliably deploy/test it out on the actual sites!
Safety is a huge concern with construction and really any operation that uses heavy equipment. I promise you they all know about Tesla autopilot running into people and will absolutely bring it up ad nauseum.
And I only say this because I think this is an interesting area to pursue so I hope you're successful with it. But a focus on safety will be of paramount importance.
I completely agree. If you have worked in the industry, I would like to talk to you and get your feedback on what I might be missing on my understanding in requirements in safety layers. Would be great if we can chat on email: contact at useflywheel dot ai
Congratulations! Looks like an interesting project!
Thanks! We enjoyed a lot while iterating the hardware and looking at hours of excavator data (mostly done by Mahimana).
I'd never thought anyone even attempting such a feat, interesting that you went all the way to mechanically actuate levers. Nice job.
But... I assume you're intending to run on extremely large sites such as highway construction, open pit mines and the likes primarily? Because my experience (if a bit dated cough) is running small 750kg baby excavators under sometimes extreme space constraints - digging trenches for telco in urban and rural areas, which often enough meant having to work with 5-10 cm distance to walls, lighting posts or other infrastructure, and directly next to workers shoving soil into the bucket. Will you add stuff like 360° camera vision, LIDAR etc. to make that safe and help a remote/AI operator, or are you planning on large sites with less danger potential only?
I can't wait for you to realize that this makes no financial sense, and you pivot to the real money maker: charging people on the internet to remotely dig holes.
When i first time 35 years ago in Siberia saw an excavator driving itself toward me without driver, i was struck like by lightning and first moment attributed it to hangover. Next moment though i saw the driver walking along the road looking for mushrooms and understood that the deep tracks in the unpaved road are like rails so that the excavator just moves along these tracks. Couple years later (this time another part of the country - a low populated part of the St.Petersburg region) my friends who were vacationing in a remote village there saw one day a large tractor passing on its own through that village, and next day guys came asking whether anybody saw a tractor without driver - the guys were drinking and details of how the tractor got away weren't clear though ...
I can relate to how you felt. Even though I developed this myself, when I saw it running remotely with no one sitting inside for the first time, it felt magical and weird. This is going to save lives and look magical and that's too much fun to put off and do something else!
The economics of this are nonsensical. The autonomy is not going to happen any time soon, navigating a construction site safely is significantly harder than navigating traffic safely.
This leaves remote operation, which just makes no sense at all. The cost of one guy going to a construction site is never going to be more expensive than retrofitting a fleet of excavators with this hardware and building a remote operating center. Additionally these should obviously not be allowed to be used at construction sites, since remote operation in such a dangerous environment adds a totally new layer of hazards. Direct communication between operators, verbally and visually, is extremely important, to operate an excavator safely.
I think, you must be technical guy. As a technical guy I share your negativity:-) While mining is already automated, the regular construction sites are too small, not scalable and retrofit costs are huge. Autonomous excavator is functional safety nightmare… it can not only drive, but has a huge moving metal part. And can dig holes and fall into these holes.
And last but not least, my car can’t keep lane and can’t drive itself safely in autonomous mode for 50 miles straight on clear day on highway. That’s state of art vision only system in 2025.
The salary of the skilled guy is laughable compared to heavy machinery costs. I was in a quarry on Tuesday. Two guys were operating 7000000€ machinery there, their salary is rounding error in the whole operation. Fuel over the year costs more.