Making Hardware Just Work

This is a draft document last updated July 10, 2003. Please send comments to Havoc.

One effort has been started to code a solution to this problem.

An interesting library someone pointed out after I wrote this is discover from Progeny. Other related projects include kudzu and of course Linux hotplug and D-BUS.

Problem Statement

This document is an attempt to think through the various elements that go in to presenting a nice, user-friendly interface to the hardware of a typical desktop system. No attempt is made to consider issues that apply to "big iron" servers, because I don't understand those issues.

Let's take a simple example of what should happen from 1,000 feet. In the following, "system" means an appropriate conglomerate of kernel and userspace features, it does not imply kernelspace.

  • I plug in my new digital camera.
  • If the camera's interface type (e.g. USB) supports it, the presence of a new device is noticed without user intervention.
  • For devices that can't be detected on plugin, there's some GUI application allowing me to explicitly probe or specify the device information.
  • The system gets whatever information is available from the device about the hardware model. Vendor, ProdID, whatever it can get.
  • The system consults a mapping from hardware model information to driver information. This mapping is created by merging three sources of information: user-provided information, OEM-provided information, and operating-system-vendor-provided information.
  • If no driver is found, the system delivers a notification to the desktop environment, and the desktop environment informs the user that the device is not supported. Alternatively, the desktop environment asks the user which driver to try and passes the answer back to the system; the system a) tries that driver and b) saves the driver information in the above-mentioned mapping so it won't have to ask in the future.
  • The system loads a driver once one has been found in the mapping, or added to the mapping.
  • One or more kernel devices are created by the driver (implying major/minor pair, and /dev file if required).
  • A notification is sent out from the system to all interested listeners that a new kernel device has been created.
  • The user's desktop environment asks the system what sort of device is represented by the new kernel device: camera, scanner, printer, etc.
  • If the system does not know, it says "device type is unknown."
  • If the device type is unknown, the desktop environment asks the user to provide it. "What did you just plug in?"
  • After the user provides this information, the desktop environment passes it back to the system. The system records a mapping from the hardware model identifying information to the device type. Next time a kernel device is created for a physical device with the same model information, the system will report its type accordingly.
  • The desktop environment allows the user to interact with the device in device-appropriate ways. For example, view images from a camera, monitor errors from a printer, or whatever.

Abstraction

Starting from the top down, what should the implementation look like? On the desktop or application level, one reality is that we will have many separate applications that care about hardware. Some of these applications will be trivial applets, others will be standalone apps such as an office suite, others will be desktop components such as the file manager.

Given these many applications, it's critical to be able to add support for new hardware models, and even new kinds of bus (USB, PCI, etc.) without modifying each indidividual application. However, obviously applications will need to understand the type of device - where "type" means something like "camera," "CD-ROM drive," and so on - the user-visible user-interesting type of the device.

Another requirement on the application level is to be cross-platform. Thus at some point in the stack, there will need to be a virtualization with "backends" for various operating systems.

For some time I've been advocating a "hardware abstraction layer." This layer is simply an interface that makes it possible to add support for new devices and new ways of connecting devices to the computer, without modifying every application that uses the device.

Devices

My proposal is that the hardware abstraction layer be a shared library API. The API would work in terms of Device objects. On Linux a Device object often maps to a kernel device, but may also map to some entirely userspace devices (printers are typically all userspace, and may even be network devices). "Device" in this API is closer to what a user would consider a device than to any specific implementation concept. The hardware library would maintain a list of devices that currently exist, and could provide those upon request.

A Device object would maintain a set of properties in a key-value mapping. It would be device-dependent which properties existed. Examples of properties:

  • Types=Camera,Storage; Types=CD-R,Storage; Types=DVD,Storage
  • Bus=USB, Bus=PCI
  • Vendor=Logitech

Not all properties come from the same place.

  • Some properties would be defined as direct mappings from some hardware feature. For example, the USB Product field would always be passed straight through to a property called something like "USB.Product"
  • Some properties would be "portable" across hardware. For example, there might be a "Description" property, which would be the best readable name for the device available. For USB, this might be a combination of Manufacturer and Product information from the hardware. PCI provides similar information.
  • Some properties would not be derived from the hardware itself, but rather from a mapping maintained by the system. For example, the "Types" property might be in this category; the system would maintain a mapping from hardware model to hardware type.
  • Some properties might combine the above two; for example, we might have a mapping that gave internationalized names for some devices; if the mapping covered a device, the internationalized name would be used for the above-mentioned "Description" field, and otherwise the library would fall back to the information from the hardware. So the "Description" property would be a function of both the hardware and the system-maintained mapping.
  • Some properties might be metadata stored on the device by the desktop environment or applications. For example an X-Nautilus-WindowPosition property storing the position of the window that's used to view storage device contents.
  • Some properties might come from translations and localizations; for example overriding the name of a device.
  • Some properties might come from blacklists and whitelists.

In short, we create a coherent view of the devices on a system and their properties by merging a set of data sources, including the physical hardware, drivers, subsystems such as CUPS, a bunch of data that comes with the operating system, any data that hardware vendors ship with their hardware, and finally data that users provide manually.

The exact set of information available to be merged depends on what operating system we're using and what kind of device we're talking about.

In pseudocode, the basic API is quite simple:

        void  list_devices (Device **devices, size_t *n_devices);
        char* device_get_property (Device *device, const char *name);
        void  device_set_property (Device *device, const char *name, const char *value);
      
It would also be necessary to have complete change monitoring; not only as the list of devices changes, but also when a device's properties change.

Some useful properties to provide, again not all would be available for all devices:

  • Device description.
  • Name of the bus the device is on.
  • Major/minor numbers.
  • /dev file(s) if any.
  • Unique device instance serial number.
  • Type of device.

Using Devices

The Device abstraction allows applications to ask questions such as "what printers are there?" and allows the desktop to include code such as:

        void
        new_device_notification_callback (Device *new_device)
        {
          if (device_type_is_unknown (new_device))
            {
               type = ask_user_for_device_type (new_device);
               device_set_type (new_device, type);
            }
        }
      

However, the next problem is how to provide functionality specific to particular types of device. For any camera, the application needs to be able to get the images from the camera. For any printer, the application needs to be able to check what print jobs it has active, and whether there are errors. For any CD-R, the application needs to be able to burn an ISO image.

It seems foolish to pile all this functionality into one giant shared library; instead, we want to build on projects such as CUPS, gphoto, cdrecord, and so forth. The missing piece, though, is a mapping from the Device object to those projects. That is, given a Device that I know is a printer, how do I locate the CUPS print queue that corresponds to that printer? Given a Device that I know is a camera that can appear as a file system, how do I mount it?

Say nautilus wants to get images off a camera; you need an API that has a function like "camera->get_images()." This API probably wouldn't live in the main hardware library. However, you need a way to get that "camera" object given a Device*. So the camera API need not live in hardware lib, but it does need to see the same set of devices seen by the hardware lib and you need a way to map from Device* to whatever a camera is in the camera API. Same holds for CUPS, sound server, whatever. Given that I can list Device* and find two sound cards, how do I specify that one of them is the output device for the sound server.

So what's needed is a way to identify a Device (remember a device often means a kernel device, with major/minor pair) that can then be passed to arbitrary subsystems such as CUPS.

The most obvious solution is to simply pass the Device* from the base hardware library around between more specialized libraries. However, this requires buy-in, e.g. the CUPS libs would need to link to the hardware library.

The other approach is to bail and break the abstraction for this purpose; i.e. allow getting at the underlying kernel device or CUPS data structure, and pass that around. This is probably the most practical thing at first.

An outstanding question is whether applications would use subsystem APIs directly, or whether the single hardware abstraction library would provide APIs for each type of device. For example, is there a Printer object in the base hardware abstraction library that uses CUPS on the backend; or do applications just use CUPS directly, after asking CUPS for the printer that corresponds to a Device*. The answer to this may vary according to the properties of the particular subsystem - is it already a portable abstraction layer, or is it Linux specific, for example.

A complication: Not all uses of a device are device-type-specific. For example, the Device object might have a generic operation to disable the device (presumably this would normally map to unloading its driver). Thus, the base hardware abstraction library might support these operations.

Implementation Goals

Here are the major areas of implementation that I see.

1. Loading the right driver

Given some raw hardware information about a device, such as its PCI or USB ID, load the right driver for the device. This involves maintaining a database mapping hardware ID information to kernel driver names and parameters.

It must be possible for an OEM to ship an addition to the database along with a product. It must also be possible for local sites to add their own overrides or provide missing mappings.

There should be a hook from this layer to the desktop, where the desktop is provided with a) whatever description of the device can be scrounged up, if only "the device you just plugged in," and b) a list of possible drivers. The desktop then asks the user to pick one of the drivers, and invokes a routine to record the choice for future reference and then load the driver. This hook would probably be in the form of a few simple functions in the hardware abstraction library.

Once a driver is loaded, we should have zero or more new kernel devices. This means that the list of Device* provided from the hardware abstraction library would be updated. So the hardware abstraction library needs some way, from inside an application, to monitor additions and deletions from the list of loaded devices. This should not be done via polling, but rather via notification.

2. Creating and maintaining /dev files

In an ideal world, /dev files are completely hidden from applications. Using them always involves knowledge of specific kinds of hardware, and thus breaks the abstraction barrier that allows us to add new device support without changing all the apps. /dev file naming may also vary by platform, and even by local site.

Currently, /dev naming is sometimes used to specify preferences such as "the default audio device" or "the default CD-ROM" (/dev/audio, /dev/mouse, etc.). Using /dev as an implementation detail for this is fine, but applications should have a way to get the default audio Device*, then play sound to the default audio Device*, without ever hardcoding the /dev/audio path in the app. Hardcoding /dev/audio means that the application must make platform-specific and devicetype/bustype-specific assumptions. These assumptions are ideally buried in a library, for all but the most specialized applications.

Anyhow, from an application standpoint, /dev naming is something that should magically go on behind the scenes of the hardware abstraction library. But it still has to be implemented somehow.

3. Tracking the list of Device* (and each device's state)

This is the primary task of the hardware abstraction library. It has several components:

  • Monitoring kernel, CUPS, etc. devices as they come and go and merging those into the list of Device*.
  • Maintaining a database that fills in the properties of a Device* that don't come from the hardware itself, such as the type of each device.
  • Understanding how to read hardware information from the various kinds of device - USB, PCI, etc.
  • Synthesizing the properties of each Device* from the hardware information, the database, user-provided information, and other available sources.
  • Exporting generic Device* operations, such as "disable this device"

4. Providing a means to use each device

This is the secondary task of the hardware abstraction library. Given a Device known to be a printer or a camera or whatever, the library must chain to CUPS, gphoto, cdrecord, or whatever is appropriate, allowing the application to use the device.

In some cases, the hardware abstraction library may offer its own APIs to use a type of device. This would be especially useful when the existing subsystem is low-level/crufty, hard to use properly, or unportable.

In other cases, the subsystem may be quite complex, something like GStreamer or CUPS; wrapping these APIs probably doesn't make sense. But there still has to be a bridge between the hardware abstraction layer and the subsystem, so they work together in a coherent way, and above all agree on the list of available devices.

Implementation Details

1. Loading the right driver

Suggestions:

  • Use the same database system here that's used for maintaining information for Device* properties. That is, use the same file formats and so forth.
  • There really must be a way for third party OEMs to add hardware ID to driver mappings, and this is really the same problem as allowing users and local sites to add them.
  • To ask the user which driver to load, D-BUS seems appropriate. It is a channel from system-space to user-login-session-space.

I don't have much opinion on this otherwise.

2. Creating and maintaining /dev files

As long as this is suitably buried beneath the hardware abstraction library, it's fairly irrelevant to desktop application developers how it works. Anything that works is great.

3. Tracking the list of Device* (and each device's state)

The hardware abstraction library should use a "model-view" architecture, where the Device object and list of Device objects make up the model, and the application implements some view of it.

It seems appropriate to use D-BUS to notify the hardware abstraction library of changes to the list of kernel devices, changes to the list of CUPS printers, etc.

Each Device object would have a backend that was either a kernel device (major/minor pair), a CUPS printer, or some other kind of device representation. Information from this backend would then be integrated with override or supplementary information from a system database, and the result would be exposed as the public API of the Device object.

4. Providing a means to use each device

The implementation here is very specific to each type of device.

Blacklists and whitelists

The core implementation idea presented in this document is to maintain a list of device objects, where each device object has a set of properties (key-value pairs).

Blacklists/whitelists of any kind simply become additional properties to be merged, or perhaps simple rewrite rules that modify a property in some way. Something like:

     if DeviceModel matches blahblah then set BrokenFeatureFoo to true
    

This is extensible to support any kind of device, device feature, blacklist/whitelist, user configuration, translations, or whatever; we just store arbitrary data associated with any device, more or less.