In Build the Ecosystem, Not Just the Robots. Examples, we suggested the “Robot Butler.”

Creating a single all-purpose household robot may be aiming at the wrong target. The ordinary home contains hundreds, perhaps thousands, of different tasks, many requiring very different physical abilities. A robot that can cook dinner, clean a bathtub, wash windows, mow the lawn, fold laundry, repair a faucet, carry groceries, change a light bulb, and make a bed would require an extraordinary collection of sensors, manipulators, tools, skills, and programming.

Humans accomplish these things with hands, legs, brains and a general understanding of today’s culture, that nature has spent millions of years developing. Reproducing all of those capabilities in one machine may be extraordinarily difficult and, more important, unnecessary.

A simpler approach is to divide the household into specialized functions. Instead of teaching an all-purpose robot to push a vacuum cleaner, build a vacuum cleaner that moves by itself.

Instead of teaching the robot to wash windows, build a specialized window-cleaning machine or incorporate cleaning mechanisms into the windows.

The same principle can apply to lawn care, laundry, food preparation, dishwashing, security, climate control, inventory, deliveries, and dozens of other household functions. Each specialized machine needs to solve only its own relatively narrow problem. This is the principle I described earlier: automate the household rather than the housekeeper.

The general-purpose robot then becomes something quite different. Think of it as the butler robot. It might have wheels, arms, cameras, and manipulators for miscellaneous jobs, but physical labor would not be its primary function.

Its most important job would be to understand what the resident wants and translate those wishes into instructions for the household’s specialized machines.

Tell the butler robot, “Get the house ready for six guests at seven o’clock,” and it might instruct the floor-cleaning robot to vacuum, the food system to prepare dinner, the climate system to adjust the temperature, the lighting system to prepare the dining room, and the lawn machine to finish its work before the guests arrive.

The butler robot would coordinate the jobs, monitor their progress, handle conflicts, and ask the human for a decision when necessary.

We already have a miniature example of this concept: the modern automobile. In that system, the human serves as the butler.

Walk toward the car with the key in your pocket and one specialized system recognizes the key, tells another system to unlock the doors, and may tell still another to turn on the lights.

Press the Start button and a sequence of specialized machines and computers prepares the car to operate. Select reverse and the transmission engages reverse while the backup camera comes on and proximity sensors begin watching for obstacles.

Shift into drive and the camera turns off, the transmission changes its function, and other systems begin monitoring speed, approaching traffic, lane position, engine conditions, tire pressure, and dozens of other variables.

As conditions change, headlights may turn on, windshield wipers may activate, warning signals may sound, and braking or steering systems may intervene.

No single mechanism does all these things. Specialized components perform specialized jobs and exchange information so quickly and routinely that the driver experiences them as one machine: the car.

The automated house would extend essentially the same principle to a much larger and more varied collection of machines, with the robot butler assuming many of the coordinating functions now performed by the automobile’s human driver.

This approach creates another problem, however. The butler and all those specialized machines have to be able to talk to one another. A homeowner should not have to buy every appliance, robot, window, thermostat, security device, and lawn machine from the same corporation. Nor should the manufacturer of the butler robot have to write entirely new software for every product ever introduced by every other manufacturer.

What is needed is a common language—a set of standards allowing machines made by different companies to identify themselves, describe their capabilities, accept instructions, report their status, and communicate problems.

There already is a precedent for exactly this kind of industrial cooperation. The Connectivity Standards Alliance developed Matter, an open smart-home communications standard supported by hundreds of companies. Amazon, Apple, Google, Samsung SmartThings, Signify, and many others participated in its development.

Matter is intended to allow smart-home products from different manufacturers and ecosystems to communicate using a common protocol. Today’s Matter already provides such functions as device discovery, provisioning, secure communications, and control.

The household-robot system I envision would take that principle much further. The objective would not merely be to let the butler turn on a light made by Company A or change the thermostat made by Company B. The standard would allow a machine to tell the entire household system what it is, what it can do, what it needs, what commands it understands, and what information it can return.

Imagine bringing home a new window-cleaning robot made by a company from which you never before have purchased anything. The butler should not require special WindowBot software. The new machine should be able to announce, in effect: “I am a window-cleaning device. I can wash, rinse, dry, and inspect windows. I require electrical power, water, cleaning fluid, and access to the window. I accept commands to start, stop, pause, resume, inspect, clean a specified area, and return to my storage location. I can report my location, progress, supply levels, equipment condition, problems, and completion.”

If both machines conform to the standard, the butler robot already knows how to employ its new worker.

Creating such standards would require cooperation among companies that normally compete with one another. A House-Robot Committee could begin with companies that already possess pieces of the necessary technology. Amazon, Apple, Google, Samsung, and perhaps OpenAI could contribute artificial intelligence, speech, vision, operating systems, and home-control expertise.

Indeed, Amazon Echo, Google Nest Hub, Apple HomePod, and Samsung SmartThings already can function as hubs for Matter devices.

Companies specializing in household robotics could provide another part of the expertise. iRobot and Roborock, for example, have experience with autonomous navigation, mapping, docking, obstacle avoidance, and specialized cleaning machines.

Appliance manufacturers such as Whirlpool, LG, Bosch, and Samsung could represent refrigerators, ovens, dishwashers, washers, dryers, and other stationary machines—which really are specialized robots whether or not we call them robots.

Building-system companies also would be essential. Schneider Electric could contribute expertise in electrical and energy-management systems; Honeywell in environmental controls and sensors; Signify in lighting; and companies specializing in locks, doors, windows, plumbing, security, and building materials could represent the increasingly automated structure itself.

Husqvarna and similar companies could represent robotic lawn and outdoor equipment. NVIDIA and semiconductor manufacturers could contribute the processors, sensors, communications hardware, and artificial-intelligence infrastructure on which the entire network would depend.

Many companies in these categories already participate in the Connectivity Standards Alliance, demonstrating that cooperation among otherwise competing manufacturers is quite practical.

The committee should not try to standardize the robots themselves. That could inhibit innovation. A company should remain free to invent a better window cleaner, another a radically different food-preparation system, and another a lawn machine nobody previously imagined.

What should be standardized are the interfaces among them: how machines describe their capabilities, how tasks are requested, how permissions are granted, how priorities are established, how status is reported, how errors are communicated, and how one machine discovers what another machine can do.

This also changes the economics of household robotics. No corporation has to solve the entire almost impossibly difficult problem of building an artificial human. Hundreds of companies can solve smaller problems at which they already have expertise.

A refrigerator company improves refrigerators. A window company improves windows. A robotics company improves mobile machines. An AI company improves the butler’s ability to understand people and coordinate machines. Improvements can be introduced one component at a time without replacing the entire system.

The result ultimately may make the distinction between the robot and the house increasingly meaningless. The mobile butler is merely the human’s principal representative within a network of specialized machines, sensors, appliances, computers, and structural systems.

Intelligence and physical capability are distributed throughout the house. One machine does not have to know how to do everything because the system knows which machine can do each thing. Further, the failure of one  machine would not require replacing the entire system, nor would it disrupt the entire system. The faulty part simply could be identified and replaced.

In that sense, the most important household robot of the future may not be a robot at all. It may be the house itself—a modular robot consisting of hundreds of specialized components, coordinated by an intelligent butler that understands what its human residents want. When the car  radio goes down, the car doesn’t stop. You just replace the radio.

The first step toward building it may therefore have surprisingly little to do with building robots. The first step may be getting the companies that eventually will build all those pieces into the same room and agreeing on how their machines will talk to one another. 

Rodger Malcolm Mitchell

The previous post, Build the Ecosystem, Not Just the Robot, discussed the incredible difficulty of creating a household robot capable of handling typical chores—cooking, cleaning, buying, etc.—that humans may find trivial. Still, their variety makes them devilishly difficult to build and program.

Brains, fingers, feet, eyes, noses, ears, skin, and taste—all attributes humans possess courtesy of nature—are hard to replicate. Even driving a car, a task humans perform with minimal thought, has proven challenging for robots.

Here are examples of solutions.

Automate the Household, Not the Housekeeper

The effort to develop household robots may be aimed at the wrong objective. Rather than trying to create an artificial human capable of performing thousands of unrelated physical tasks, in various environments, it may be far easier to automate the details of the household itself.

The general-purpose robot would become less a maid, cook, gardener and handyman than a butler: a mobile coordinator connecting the human with a collection of specialized automated systems.

A modern automated factory does not ordinarily employ one fantastically capable humanoid robot that walks around welding, painting, machining, inspecting, packing and driving a forklift.

It distributes those functions among machines optimized for particular jobs and coordinates their activities. A highly automated home could operate according to the same principle.

The “Roomba” offers a simple example. One approach to automated floor cleaning would be to teach a humanoid robot to find a vacuum cleaner, grasp its handle, operate its controls, maneuver it around furniture, determine what it has already cleaned, and return the vacuum when finished.

The simpler solution is a vacuum cleaner that moves by itself. The specialized machine needs to understand only its specific job and the part of the environment relevant to that job.

The principle can be extended throughout the house. Don’t give the general-purpose robot a scrub brush and teach it to clean a bathtub; develop a bathtub-cleaning system.

Don’t give it a squeegee and teach it to wash windows; incorporate cleaning mechanisms into the windows or provide a specialized window-cleaning machine.

Don’t teach the robot to mow the lawn; give the lawn its own machine. Don’t teach it to load a dishwasher; automate the entire food-service and cleaning process.

The general-purpose robot—think of it as a butler robot—then doesn’t need thousands of highly refined motor skills. It needs to understand human intentions, communicate those intentions to specialized systems, coordinate their activities, supervise results and deal with exceptions.

Physical manipulation remains useful, but it becomes one capability among many rather than the foundation of household automation.

Consider food shopping. Even sending a robot to a store may preserve an unnecessary human-era activity. If every food item has an electronic identity and the refrigerator and pantry maintain continuous inventories, the house already knows what it contains, how much remains and the household’s historical consumption. It can predict that milk will run out Tuesday afternoon or that only two eggs remain.

Routine shopping therefore can disappear. The household system can compare suppliers, prices, quantities and delivery times and automatically replenish ordinary items within limits established by the resident. The system can refer unusual purchases or significant price changes to the resident for a decision.

The store, which can resemble an “Amazon-style” warehouse, can deliver standardized containers directly to the house’s receiving dock, where refrigerated items are routed to refrigerated storage, frozen foods to frozen storage and dry goods to their appropriate locations.

Inventory would be updated automatically.

No robot needs to drive anywhere. No car needs to park. Nobody needs to walk through aisles. The shopping trip was merely another traditional human task whose underlying purpose—maintaining an appropriate inventory of food—can be accomplished without the task itself.

Meal preparation can be treated similarly. A resident might say, “I want steak, mashed potatoes and peas at six o’clock, with approximately 300 calories and 12 grams of protein.” The butler robot does not peel potatoes or stand over a broiler. It converts the human request into objectives, preferences, constraints, and a deadline, then communicates them to the household food system.

The food system selects appropriate ingredients from inventory, determines quantities, prepares and cooks the components at appropriate times and temperatures, and produces the finished meal.

If the requirements conflict—for example, if the requested steak portion by itself would exceed the calorie limit—the system reports the problem to the butler robot, which asks the resident whether to reduce the portion or increase the calorie limit.

This is where general-purpose artificial intelligence is particularly valuable: between human intention and specialized machinery. The human specifies the desired result. The AI interprets that result, resolves ambiguities, coordinates the necessary systems and reports problems requiring human judgment.

After the meal, there is no reason to assume that the traditional sequence of clearing dishes, loading a dishwasher, unloading it and putting dishes into cabinets must survive. Reusable dishes might move automatically into an integrated cleaning-and-storage system.

Alternatively, dishes and utensils might be fabricated when required from recyclable materials and returned afterward to the material supply. The table itself might be built and programmed to perform some of these functions.

The important point is not which particular solution ultimately prevails. Tasks should not become the one butler robot’s automated task.

The correct first question is not, “How can we build and program this robot to fold laundry?” It is, “What is the objective of folding laundry?” If the objective is to store clean clothing conveniently, perhaps a different storage system eliminates folding altogether. Or at worst, the storage system would be programmed and built specifically to fold laundry, an easier build than trying to do it with an all-purpose robot.

The objective of shoveling snow is not shoveling snow; it is keeping necessary outdoor surfaces usable. The objective of replacing a roof shingle is not manipulating shingles; it is keeping the building weatherproof. The objective of washing a plate is not washing a plate; it is providing a clean surface to eat from.

This suggests a fundamental rule for household automation. Don’t automate the task. Identify the task’s purpose and automate that specific purpose.

The resulting house becomes a network of specialized systems. Cleaning machines, food-storage systems, cooking equipment, laundry systems, waste processors, environmental controls, security equipment, delivery systems, garden equipment, structural monitors and repair systems continually exchange information.

The mobile butler (the primary robot) is one participant in that network, serving mostly as the resident’s general-purpose representative and coordinator.

Information can travel in both directions. The refrigerator can tell the food system that a steak should be used soon. The food system can tell the butler that steak therefore would be a sensible dinner choice. The butler can ask the resident, “How about steak tonight?” A four-word answer can initiate dozens or hundreds of coordinated operations.

The architecture also offers an important reliability advantage. If a single extraordinarily sophisticated humanoid robot performs every household function, that robot’s failure may disable virtually the entire automated household.

In a distributed system, a window-cleaning machine failure means the windows remain dirty until that component is repaired. Food preparation, laundry, floor cleaning, security and package delivery continue functioning.

Specialization also permits incremental technological improvement. A homeowner need not replace a fantastically expensive general-purpose robot because someone has invented a superior floor-cleaning technology. The floor-cleaning component can be replaced independently. The same principle has made industrial automation practical and continuously improvable.

Eventually the distinction between robot and house may become increasingly artificial. The building itself contains sensors, communications, storage, machinery and specialized automated systems. Mobile robots provide those capabilities requiring mobility. Fixed machines provide capabilities that do not require mobility. The intelligence coordinating them may be distributed throughout the system.

The house, in effect, becomes a modular robot, with each replacable module trained for a specific task..

This approach also could make household automation achievable sooner. An artificial human capable of successfully manipulating virtually every object and tool found in an ordinary house presents an extraordinary robotics problem. A coordinated collection of machines, each optimized for a relatively narrow function, presents many smaller and more manageable engineering problems.

The central research objective therefore should not be “Build the perfect household robot.” It should be “Automate the household.”

For every existing household activity, ask three questions.

1. What is the actual objective?
2. Does the traditional task need to exist at all?
3. If it does, should a general-purpose robot, a specialized machine, or the house itself perform it?

The result may be a paradox: a far more automated home that requires a far less capable humanoid robot.

Instead of making a robot sophisticated enough to handle the extraordinary complexity of a house designed entirely for humans, divide the system into tasks and make the whole system sophisticated so the robot doesn’t have to be.

To begin the evolution, relevant industries should coordinate their efforts so they can determine some initial standards. Companies that make stoves, washing machines, windows, roofs, lighting, floor cleaners, building materials, tools, and robots, etc., should begin working together to create the “smart house.”

That is where the future lies, not with an all-purpose robot, but with a society assisted by automation.

Rodger Malcolm Mitchell

Build the Ecosystem, Not Just the Robot

In “Grasping the World,” from the October 2026 issue of Scientific American Magazine, author Adam Rogers highlights the immense challenge of creating an all-purpose robot that can operate in a world designed for humans.

While a robot can master a backflip by focusing on its own body mechanics, tasks like scrambling eggs, folding laundry, or handling grapes are far harder because they require the robot to navigate an infinitely variable external environment.

Google DeepMind is working to solve this through AI that generalizes skills across different tasks and robot bodies, yet even seemingly trivial human activities remain a hurdle—in one test, a robot successfully used a dustpan only 32 percent of the time.

We may be approaching the problem from the wrong direction.

Most household-robot research treats the environment as fixed. Because our houses are filled with objects designed over centuries to accommodate the human body—doorknobs, light switches, utensils, and tools—we assume the challenge is to build a robot smart enough to operate all this equipment.

Instead, we should work both sides of the problem: Don’t build robots that can function in our existing houses.

Start building houses that work with robots.

Consider something as simple as hanging a picture. A human installs screw eyes and wire on the frame, decides where the picture should go, handles a tiny hook and nail, pounds the nail into the wall, lifts the picture, catches the wire on the hook and straightens the frame.

Teaching a robot to perform all those operations would require considerable visual, tactile and manipulative ability.

But why should a robot hang a picture that way?

A robot-compatible wall might contain an invisible mounting grid. Picture frames could have standardized attachment systems. The robot merely places the picture against the desired portion of the wall; the mounting system grabs it and levels it.

An extremely difficult robotics problem has almost disappears—not because the robot became smarter, but because the environment cooperated.

The same principle applies throughout the house. We should not teach a robot to fold laundry. We should solve the clothing-cleaning-and-storage problem, perhaps designing clothing and storage systems that make folding unnecessary or do the folding themselves.

We should not teach a robot to load a dishwasher. We should solve the eating-vessel problem. Dishes might remain in an integrated cleaning-and-storage system. Inexpensive dishes might be fabricated ad hoc, and their material recycled afterward.

We should not teach a robot to climb a ladder and replace a roof shingle. We should design roofs with standardized, machine-accessible, self-replacing modules.

We should not teach a robot to shovel snow. The objective is to keep necessary outdoor surfaces usable after snowfall. Snow shoveling is merely the historical human solution to that problem.

This suggests a fundamental rule: Don’t automate the task. Identify the task’s purpose, then redesign the system to accomplish it.

That changes the meaning of the “smart house.” Today a smart house generally means a conventional house with added intelligence: thermostats, lights, locks, cameras and appliances that communicate electronically.

The house we are describing would be fundamentally different. Its physical architecture would be designed from the beginning for cooperation among humans, stationary machinery and mobile robots.

And the concept would extend far beyond the house.

Consider food. The objective “prepare dinner” conceals an enormous chain of activities. Menus must be selected. Recipes must be developed. Ingredients must be inventoried, ordered and paid for. Food must be delivered, accepted, stored, retrieved, prepared, cooked and served. Leftovers must be stored. Eating utensils and vessels must be cleaned or recycled. Waste must be processed.

Every link could be redesigned. Why should groceries arrive in bags designed for human hands? Why should a refrigerator consist of shelves behind a large door? Why should mustard come in a jar that a robot must grasp, open, manipulate, close and replace? Why should a delivery person leave packages on a porch designed around a human opening a front door?

The robot-compatible house eventually implies the robot-compatible store, warehouse, delivery vehicle, package, food container and waste system.

The same would be true of the duties we now assign to a maid, cook, handyman, gardener and butler. Instead of reproducing their physical actions, identify their objectives: keep the house clean; provide meals; maintain the structure; maintain the grounds; manage goods entering and leaving the household. Then design the entire system around achieving those objectives.

This immediately creates a chicken-and-egg problem. Why would a food manufacturer produce robot-compatible containers when few houses can use them? Why would builders install robotic maintenance systems when few suitable robots exist? Why would robot manufacturers design for interfaces that houses don’t yet have?

There probably can be no master plan. The possibilities and their collateral effects are too numerous. We likely will slink into the robot ecosystem.

The first innovations will be things useful even without sophisticated robots: standardized machine-readable packaging, automated package receivers, modular plumbing, self-diagnosing appliances, machine-accessible service spaces, improved household sensors and standardized attachment systems. Older houses will acquire adapters. New houses will incorporate successful ideas directly.

Manufacturers will discover that one innovation makes another profitable. Competing systems will appear; some will disappear; standards gradually will emerge.

The automobile developed much the same way. The automobile brought roads, garages, gasoline stations, parking lots, traffic lights, repair shops, insurance systems, and now, charging stations. Society didn’t merely invent the car; over decades it created an automobile ecosystem.

Household robotics may follow the same path. This is where AI may have another, even more important, role. AI needs not merely to control the household robot. It can help design the ecosystem.

Change the food container, and you may affect manufacturing, shipping, retailing, refrigeration, robotic manipulation, household storage, cooking and recycling. Change the wall system, and you affect construction, electrical wiring, plumbing, decoration, repair and robotic access.

Each solution produces collateral problems and opportunities elsewhere.

The number of interactions approaches what we call Universal Engineering: Don’t merely ask whether a proposed solution works. Ask: If we do this, what happens to everything else?

Ultimately, we may discover that the household robot itself was the wrong unit of thought. A house could contain thousands of sensors reporting temperature, moisture, structural stress, electrical loads, water flow, food inventories and equipment condition.

Specialized machines could clean floors, handle food, maintain the grounds or inspect service spaces. A mobile general-purpose robot might perform only those tasks requiring mobility and manipulation. In that arrangement, asking where the robot ends and the house begins becomes increasingly arbitrary.

The house is part of the robot, and the robot is part of the house.

Rogers describes the extraordinary effort required to make a robot understand and manipulate the complicated world humans have created for themselves. We should continue that work—while also making the world easier for robots to understand and manipulate.

Biology offers an interesting precedent. Organisms did not evolve independently and then get dropped into an unrelated environment. Organisms and environments changed together.

Robotics eventually must do something similar. The objective shouldn’t merely be: “Build the perfect robot.” It should be: Build an ecosystem in which robots work.

That last distinction—task versus purpose—is one of the idea’s strongest parts. It prevents us from spending enormous effort teaching machines to reproduce human methods that exist only because of the peculiarities of human bodies and human history.

Finally, we focused on the house. But the same thinking can apply to the office, and to some degree, already has applied to manufacturing, which actually provides the proof of concept for the whole idea.

Factories long ago recognized that forcing a robot to replicate human movements in a traditional workspace is inefficient and unnecessarily complex. Instead, factories evolved to meet automation halfway by redesigning the environment to cooperate with the technology.

By implementing assembly lines, standardized parts, fixed-position jigs, conveyors, and machine-readable identifiers, manufacturers transformed the factory floor into an integrated system where the infrastructure itself facilitates robotic performance.

The office is somewhere in between. We still design offices largely for human bodies—desks, keyboards, screens, filing systems, meeting rooms—but the information environment already is being redesigned for machines.

Electronic documents eliminated much physical filing. Databases eliminated countless searches through cabinets. Email changed message delivery. Calendars, accounting systems, workflow software and AI increasingly allow information to move directly from machine to machine without somebody physically carrying or retyping it. Electronic meeting systems have replaced meeting rooms.

Offices have begun to focus on the overall goal, which may not require people to travel to a central location. That is why “work-from-home” and the equipment that enables it have gained some traction.

For generations, we implicitly defined “work” as something like: travel to a building, sit at an assigned location for eight hours, interact with coworkers there, then travel home. Once that became customary, we designed an enormous supporting system around it—office towers, desks, elevators, parking garages, commuter trains, highways, downtown restaurants, business clothing, and so forth.

But none of those is the objective. The objective is to produce the work.

As communications technology became better, we asked the equivalent of our dustpan question: Why are we transporting the worker? For many jobs, what really needs to move is information.

So instead of making transportation more efficient—better highways, faster trains, larger parking garages—some businesses have eliminated the transportation task altogether. Computers, broadband, video conferencing, shared documents, cloud storage, electronic signatures and collaborative software constitute the environmental redesign that makes that possible.

Of course, some work benefits from physical proximity, and some requires it. But that simply means the system should distinguish between activities whose purpose requires colocation and activities for which colocation was merely the traditional method.

That gives us an even more general Universal Engineering rule. Don’t begin by asking how to perform an existing task more efficiently. First ask why the task exists. Sometimes the best robot won’t be one that has learned how to use the dustpan. Sometimes the best solution is no dustpan. And sometimes the best commute is no commute.

Our principle isn’t really about houses, offices or even robots. It is much broader.  When automating a human activity, don’t design a machine that only can operate in the existing environment. Redesign the environment, the tools, the objects and the process together with the machine.

Factories are farthest along because they are controlled environments, and economic incentives for automation are enormous. Offices are undergoing the same transformation primarily in their informational architecture. Homes lag because they are extraordinarily diverse, cluttered, personal and filled with objects inherited from generations of human-centered design.

Hospitals would be another fascinating example. So would hotels, restaurants, warehouses, farms, construction sites, airports and stores. A hospital designed simultaneously with its robots might look very different from a present hospital into which robots subsequently are introduced.

And eventually the idea extends to the city. Sidewalks, curbs, doors, elevators, delivery points, garbage collection, traffic systems and buildings could acquire standardized machine interfaces.

So even “Build an ecosystem in which robots work” is slightly too narrow. The larger engineering principle might be, “Don’t adapt the machine to the system. Identify the goal; then design the machine and the system together to reach that goal.”

That would require diverse industries to work together, rather than each trying to solve disparate problems.

That is the Universal Engineering principle.

Rodger Malcolm Mitchell

Here is a beautiful, well-written post I wish I  had written.