Aros/Developer/IODeviceDriversDev

Introduction

AOS does call Forbid() when dos list is locked and Permit() when unlocked. (also mentioned in Guru Book that it is 1.x compatibility "feature")

Currently there is following comment:

/* This came from MorphOS source code, however looks strange.
   Commented out but left for reference. 
    if (dl)
    Forbid(); */

Remove comments or put it inside #ifdef _mc68000 or do nothing? It is AOS semi-undocumented feature after all. I think we should Forbid() on LDF_WRITE, but leave it open on LDF_READ.

Time to move handlers from DEVS: to L:. They are now packet handlers just like AOS handlers in L:. End of June 2011, now with DOS packets in place, file handles and locks aren't the same thing anymore. Code like this:

    fdesc *desc = __getfdesc(fd);

    if (!desc)
    {
        errno = EBADF;

    return -1;
    }

    return __stat(desc->fcb->fh, sb);

in compiler/clib/fstat needs to create a lock from the file handle. What's the best way to do it, NameFromFromFH and then Lock? Do we need a completely different approach for parts of clib? There is LockFromFH/DupLockFromFH() and OpenFromLock() - which covers both directions. Not needed, we can use ExamineFH and NameFromFH inside of __stat().

fib_FileName stores as AROS_BSTR. So if AROS_FAST_BSTR is defined, then it's a C ASCIIZ string, and if not, it's a BSTR.

Since fib_FileName and fib_Comment are guaranteed to be BPTR aligned, it's perfectly safe to use the AROS_BSTR_* macros on MKBADDR(&fib_Filename[0]), which will wrap it all for you.

Devices

Suggest to rewrite these two ps/2 bus drivers and then use timer.device for all required delays (added a lot of busy loop delays in the code. Especially in interrupt code...).

Separate serial mouse driver.
Merge PS/2 mouse driver with keyboard driver (they use the same low-level functions and need to be linked together, this is why i

still link x86-64-pc board support package into ELF).

Under AmigaOS 3.1 an application would query the printer.device which would read the printer prefs file(s) and talk to the selected printer driver (which was custom for each printer) for that info and give it to the application program. Along with consulting its own prefs for page setup, etc., the application program would then use some controls given by the printer device to control the actual printer driver and feed it the right commands, binary bitmaps or ASCII text data to do the print job. I'd recommend starting with creating for AROS standard (ASL like) printing support requesters and developing a new printer prefs file standard and printer.device that could integrate well with CUPS, Ghostscript, etc., which are currently developed and supported open software projects.

create a printer.device OS31 compatible - The device has to call the appropriate functions in the printer driver e.g. init(), dospecial(), render(), transfer(), density() and deal with various CMD calls e.g. CMD_FLUSH, RESET, START, STOP, WRITE, DUMPRPORT, PRTCOMMAND, QUERY, RAWPRINT etc. and call the appropriate functions, also try and make it RTG compatible as well.
create a spooler task in it
create a driver to test my network printer

Then add:

a spooler monitor
create the prefs app (maybe with printer.device v44 prefs messages, don't know much yet)
a driver calling ghostscript (if the actual gs port is usable)
a driver for my PS printer and add support for the USB link
a driver for my PCL printer (the same one)
the idea of Sonic: gfx device context for printing
a driver calling something else like cups binaries

Postscript      Text  Graphics      
  |               |        | 
  |            PS Encapsulation 
  |                  |          
  |            Ghostscript Core      Rendering Section
  |                  |
  +------------------+ 
  | 
Print Queue                     
  |
  +----+-----+-----+         Printing Section 
  |    |     |     |
Net:  Par:  USB:  Ser: (???)

Support needed:

The page setup would have standard options (plus user customizable ones) for selecting orientation, page sizes and media types. (a starting point would be the ones listed in the [www.pwg.org/ipp/index.html IPP] (Internet Printing Protocol) standards documentation.

Print options should have place to select which pages of a document to preview, print or print to disk as a file (all, current, range) print file types could be PDF, SVG, PNG, JPEG. It should have settings for N-Up, duplex, add job-sheets, finishing features like staple, punch, cut(roll type media).

Color Management with queuing and job management.

Implementing drivers that can render to bitmaps and feed it to printers using umpteen different formats to handle low end inkjets that don't support PCL or PS, many of which don't have documented protocols (e.g. you'll likely have to reverse engineer CUPS drivers anyway)

Filesystems

Improved hostdisk.device design, merged some common code. Implemented handling 64-bit image file length on Windows. Use flat LBA by default instead of fake geometry which might not fit.

IIRC single-block cylinders are a problem. I think some partition/FS structures need at least two blocks per cylinder, maybe more. See Poseidon massstorage.class, where I ran into problems with this.

I'll check Poseidon code. However, at the other hand, flat LBA addressing seems to be the only reasonable choice to coexist with other OSes who already ignore cylinder alignment for a while. I wonder why AmigaDOS creators use cylinders at all, since on both low and high level AmigaOS uses LBA addressing. Filesystems just do cyl * sectors * heads. This is more likely Poseidon flaw.

Booting

according to the RKM's scsi.device section, it appears that device drivers themselves need to parse the RDB and generate bootnodes, which is quite different from the current AROS methods, where "Boot Strap" takes care of RDB parsing.

So, here's a plan. It should be more like AOS, and will result in 'hotplug' support for RDB volumes - ie RDB CDROMs and CompactFlash devices. So basically the code in bootstrap that recursively scans a disk's partition tables will be moved into a function in partition.library that will be called from device drivers? That sounds good, and would also eliminate the duplication of this code in the mass-storage class.

Some points:

How will the DOS device for a partitionless volume be handled? Currently, ATA devices start out as HDx and ATAPI devices as CDx. If the HD has a partition table, HDx is removed and replaced with whatever partitions are own IO. In the same way. If there are no partitions, then a 'bootblock' style node is created (HD0), otherwise, you get just the partitions. partition.library will be handling all the DOS details for block devices - they just need to call the function on disk change.

It appears that most of our filesystems *do* handle ACTION_INHIBIT and ACTION_DIE (mostly) properly.

partition.library -------------

Add a 'BOOL MakeDosPartitions(CONST_STRPTR device, IPTR unit)' to

partition.library

 - Enumerates through all partitions (RDB or MBR), and
   MakeDosNode() a list of DOS nodes (dnlist), generating unique
   DOS device names if needed. (ie "HD0.1" if "HD0" exists)

The bootstrap already includes code to generate DOS device names where necessary, so there should be a good reason before we change the format used.

 - If the new partition list doesn't match an old dnlist in the
   partition.library's current 'everything I've seen since boot'
   database:

   * Calls RemDosEntry() on all DeviceNodes in the old dnlist
     for this device/unit (if any).

   * Removes all the old dnlist entries from Expansion->Mountlist

   * Remember the new dnlist as one seen by partition.library

 - But if it *does* match an existing list, use that instead of
   the new one we just made.

 - Call AddBootNode() on all the DeviceNodes in the new list,
    with appropriate (bootable > -128, or nonbootable = -128) priorities.

every foo.device that can have an RDB/MBR/GUID parition ------

Modify ata.device, usb massstorage.class, etc:

- calls Partition/MakeDosPartitions(..)

  on every diskchange event.

trackdisk.device ----

Create DeviceNode entries with DosType = 0, no handler, and

 DE_BOOTBLOCKs etc. set correctly for a BootBlock style BootNode.

AddBootNode(5, 0, dn, NULL)

Boot Strap ----

Initial boot-device selection menu (from dosboot.resource) is now moved to here, and can be used to adjust the order of the entries in ExpansionBase->Mountlist, among other things.

Since the partition.library maintains the device's partition to volume mapping, we can remove the partition code from strap.

1) Scans through ExpansionBase->Mountlist, and:

 * If not a valid BootNode (node type is not NT_BOOTNODE), ignore it.

 * If BootNode->bn_Pri == -128 (non-bootable), ignore it.

 * If any of the following are true, ignore it:
    - (dn = BootNode->bn_DeviceNode ) == NULL
    - (fssm = BADDR(dn->dn_Startup)) == NULL
    - (de = BADDR(fssm->fssm_Environ)) == NULL

 * bootblocks = (de->de_TableSize < DE_BOOTBLOCKS) ? 0 : de->de_BootBlocks;

 * If the node is a valid BootPoint node (bootblocks == 0): (m68k only)
    - (cd = (struct ConfigDev *)BootNode->bn_Node.ln_Name) != NULL
    - (da = cd->cd_Rom.er_DiagArea) != NULL
    - (da->da_Config & DAC_CONFIGTIME) != 0

    -- If *all* the above is true, then:
       if da->da_BootPoint(configDev, register A2 = bn) returns,
           then booting from this device failed.

 * If the node is a valid BootBlock node (bootblocks > 0): (all archs)
    - OpenDevice(fssm->fssm_Device, fssm->fssm_Unit, &io, fssm->fssm_Flags ) == 0 
    - Read a bootblock as described in the RKM BootBlock section,
      and verify that the checksum is correct.

    - If de->de_DosType == 0:
       * Update the BootNode's de->de_DosType with the DosType in the
         bootblock

    - If de->de_DosType != bootblock's de_DosType
       * Ignore this device, continue to the next one in the Mountlist

    - m68k ONLY: Actually call the bootblock code
      If the BootBlock's init() routine returns, assume booting from this
         device failed.

    - non-m68k ONLY: Simulate calling the bootblock code
      - InitResident(FindResident("dos.library", BNULL));

2) No valid bootblocks? Then try to start up dos.library, and see if there are any 'DOS bootable' nodes.

 * InitResident(FindResident("dos.library", BNULL));
   /* If we return here, then DOS couldn't mount a SYS: volume */

3) Show a cycle of the boot animation, and go back to (1)

dos.library -----

The major change here is that dos.library *can fail*, and *can exit* with a return code of FALSE if it doesn't find a SYS: volume.

After basic DOS library initialization:

 1) While ExpansionBase->MountList is non-empty {

     * Remove the first entry (highest priority) off of the MountList

     * If a valid boot node, try to DeviceProc() it.

     * If DeviceProc() succeeds, this is the SYS: node - exit the loop

     * Otherwise, add the node to a temporary list (bn_unbootable)
   }

 2) If no SYS: node:
       * Re-add bn_unbootable to the Mountlist
       * clean up DOS allocated memory
       * return FALSE from this init of dos.library - This is safe, as no DOS volumes have been mounted

 3) Otherwise, we have a bootable SYS: node!

 4) (non-Amiga) If the SYS:AROS.boot file exists and is *NOT* valid for this machine, Intuition up a requester to tell the user about it, and allow the user to reboot or continue into the weeds of danger.

    - Since we can't safely dismount SYS: at this time, there's no easy way to handle this case without a reboot.

Finding a partition with a non-matching AROS.boot file isn't an error condition. We need to support 32-/64-bit dual boot systems. That is not possible at the moment, as far as I can tell. A 32-bit Exec won't run 64-bit code, and a 64-bit Exec won't run 32-bit code. The ABIs are completely different.

Supporting that would be.. an interesting undertaking, to say the least, and not within the scope of this proposal.

 5) Add bn_unbootable to the Mountlist - NOTE: The SYS: device node will *not* be in the Mountlist anymore

 6) For each node in the Mountlist:

    * If a valid bootnode (NT_BOOTNODE and bn_Pri > -128):
        - If ADNF_STARTPROC is set, DeviceProc() it.

 7) Execute the dosboot.resource
     DOSBootResource = FindResource("dosboot.resource")
     InitResident(DOSBootResource, 0)
     /* If we return from here, ALERT! */

dosboot.resource ----

This resource is no longer auto-initiated.

All the boot-menu stuff has been moved to "Boot Strap"

All the mounting has been moved to dos.library

All we have to do is create the initial assigns, and start the shell

Maybe we move this into dos.library, too?

Create "Initial CLI" process:

    Create initial assigns

    InitCode(RTF_AFTERDOS, 0)
       - This is, I believe, more like AOS, in that the initial
         assigns and CLI are up during RTF_AFTERDOS, and
         RTF_AFTERDOS code is called in the "Initial CLI" context

    Execute S:Startup-Sequence or Shell

RemTask(NULL);

There are several 'malicious edge cases' that can only be solved by making sure that all handlers can properly deal with ACTION_DIE, and stale FileLocks and FileHandles when they are restarted.

Not sure of the need for a partition.device. There is already a mechanism to handle hot swapping of USB drives from one port to another while preserving the volume's locks and notification requests (I don't think it matters whether the two ports are on the same controller or not). It works the same way as the mechanism that lets you swap a floppy from DF0: to DF1: etc. in AmigaOS. This is already implemented for fat-handler. The idea here is to make a single code-point that handles this for all existing filesystems, without having to have each filesystem hook into the device's diskchange mechanics.

If the handler gets a new message (ie trying to write to an existing filehandle), it's raw block IOs would go to the partition.device unit for that volume, and if Intuition isn't available, that IO would block (ie WaitSignal()) until the volume is remounted.

All of our handlers should support ACTION_INHIBIT and ACTION_DIE properly so that will the handler will not think the volume is still present when it's actually been removed.

Alternatively, partition.library could issue a pop-up (ie "The volume BigDog: is no longer available. [Wait for Volume] [Return Error to Application]") and either Waitsig() for the volume to come back, or return an IO error to the filesystem.

In either case, yes, the handler will probably stick around.

If we want to make a big push to support ACTION_INHIBIT in all the filesystems, then we don't need the partition.device proxy device. .. but, as they will remain bound to the same Exec Device/Unit when uninhibited, we will still need the partition.device proxy to handle hotplugging a partition set (since the USB stick or AHCI hotplugged drive could move to a different unit or even different Exec device). [ACTION_INHIBIT is kinder, as this should prevent the pile up of 'offline' volumes if we attempted to ACTION_DIE ejected partition sets' handlers while they have Locks or FileHandles open. At least AFS appears to be able to resume volumes that it finds again.] The partition.device proxy is not needed, just the automatic sending of ACTION_INHIBIT as needed on disk changes.

Was it ever needed in conjunction with offline volumes in AmigaOS? Yes, it's critical to the implementation of the DosList volume management in AFS. See the ACTION_INHIBIT: section of afs/main.c, and follow the code paths.

What's wrong with having offline volumes? They can stick around after their handler dies and be adopted by another instance of the handler when re-inserted. Fat-handler and massstorage.class serve as a good proof-of-concept IMO, as all this hot-swapping and off-line volume stuff works/worked fairly well (I haven't tested this functionality since the packet switchover).

[Yes, you can eat up all your memory by hotplugging a zillion different USB sticks, even with ACTION_INHIBIT working, but I consider that a non-interesting edge case right now. If it becomes interesting, we should make sure that at least afs-handler and fat-handler properly deal with ACTION_DIE].

Steps

When an Exec block device (ie trackdisk.device unit 0) is first mounted, it is assigned a Dos Device node (DLT_DEVICE) that has a handler (ie FFS), and that handler knows that it will, until it ACTION_DIEs, be attached to the Exec device specified in its DosEnviron (ie trackdisk.device, unit 0 (for brevity, 'td-0')
Suppose a floppy is inserted. At this time, the td-0 handler (FFS) probes the floppy and, having detected a FFS disk, creates a new DLT_VOLUME for it, and sets the volume's dol_Task to the handler's dol_Task. Don't think FFS determines if the filesystem is FSS, think it "assumes" it is - and if it cant read it, it assumes it just hasn't been formatted or is corrupt.
New locks are attached to the DLT_VOLUME's dol_LockList. (maybe lazily, but this is the theory). Sort of. They are usually only attached to dol_LockList once the volume goes offline (the media is removed).
Suppose the floppy is ejected. At this time, the td-0 handler (FFS) probes the floppy device, finds no disk present, and performs ACTION_INHIBIT/TRUE on itself. The ACTION_INHIBIT sees some locks open, sets the DLT_VOLUME's dol_Task to NULL, and leaves the DLT_VOLUME in the DosList. More or less. The handler receives indication of the floppy's absence with the commands Toni mentioned. It is at this point that the volume's outstanding locks are attached to dol_LockList.
If IO is attempted on a lock/filehandle on the ejected DLT_VOLUME, Dos detects that the Lock's fl_Volumes' dol_Task is NULL, and pops up the 'Please insert volume XXX' requester. Version 1.0 had a difference here between DOS and FS. If DOS needed a file from Volume Vol:, but it found it wasn't present, it would ask the user to insert it anywhere. But if the FS was trying to locate another block on a disk, then it would ask to replace the disk in that specific drive. (Yes, I think, it did just say "disk", rather than giving its name, which was a major irritation if you'd just done three disk-swaps.) I can't recall how far this was improved over time, though.
1. A new floppy is inserted. The td-0 handler performs the same action as (2). Something will have to inhibit DOSFALSE, or nothing is going to happen. If td-0 is waiting for reinsertion, it'll now probably return the requested data. The handler is actually waiting for a IOHF_MEDIA_CHANGE, and will rescan the device to determine what the new volume is, at least that's how AROS has it implemented. In AROS the probing is done in the FFS handler itself on media insertion. From the best I can tell, AOS does the same.
The floppy originally on td-0 is now inserted in td-1. After probing, the td-1 handler finds the same volume on the DosList, and sets the volume's dol_Task to td-1's dol_Task. The original volume is now available. Yes. Additionally, the td-1 handler grabs the volume's locks from dol_LockList. Available to the DOS, that is. You can open the files on that volume again. If td-0 had a file open on it, it will continue to ask to reinsert it (unless AOS or AROS improved on that). Handler uses device driver's TD_ADDCHANGEINT/TD_REMCHANGEINT (older code used TD_REMOVE, or in worst case polls) i/o commands to detect media changes.

The basic summary is that the DOS DeviceNode is bound to that device for the lifespan of the DeviceNode (until it receives ACTION_DIE), and that it manages the generation, detection, ejection, and resurrection of any volumes it sees, _of_its_filesystem_type_, _for_that_exec_device_.

If so, here's where I think this model breaks down:

1) When an Exec RDB capable block device (ie ata.device 0) is first seen, the partitions are scanned, and DOS devices are created for each partition, and assigned handlers. Each of those handlers is bound to the Exec device, unit, and block ranges specified by their DosEnviron vectors.

2) Each of those DLT_DEVICEs expects to be able to communicate with its underlying Exec device until it receives either a ACTION_INHIBIT or ACTION_DIE.

3) Assume now that the multiple partitions have created their DLT_VOLUMEs, and we have some open locks on some, and others have no locks.

4) The ata-0 device is then ejected. Each of the existing DOS Device handlers will get IOHF_MEDIA_CHANGE events, and detect that no media is present, and, just like case 4, set up their volumes to be dismounted or 'not available. They shouldn't get MEDIA_CHANGE; they should get INHIBIT. The medium that's changed is not theirs; theirs just disappears because the medium it's on is changed. Compare with diskfiles.: You have mounted a diskfile from an USB-stick. You pull the stick; the diskfile doesn't suddenly get a different volume inserted, it just can't access its volume any more. Reinsert Volume AOSDMS: into device DFile1:.

Points 1 to 4 are correct, but additional steps may be taken during 4, depending on the type of drive we're talking about. Assuming we're talking about a hard drive on a hot-pluggable interface, e.g. a typical USB hard drive or flash drive. For USB, step 4 corresponds to the HD's USB plug being removed from the system. All handlers using the HD would be sent ACTION_DIE. Those handlers with outstanding locks would put them on their volume's dol_LocksList before dying. After the handlers are gone, the Exec block device for the removed HD will be destroyed.

If we consider a handler returning DOSFALSE for ACTION_INHIBIT or ACTION_DIE when it has no open locks a bug.

Or is it an even stronger requirement, that (so long as there are no in-process IOs) a Dos Device handler should always properly handle ACTION_INHIBIT and ACTION_DIE, and migrate their locks to the offline volume, and a new handler should be able to resurrect those locks once the volume comes back online?

If there are active IOs, and the handler returns DOSFALSE for action ACTION_INHIBIT and ACTION_DIE, then the handler should hang around, and wait for the device to be reinserted, correct?

5) A new MBR device is inserted. The existing DOS handlers, which still are allocated to the old partition layout, see the OHF_MEDIA_CHANGE, and try to probe for new volumes on the new disk in their old ranges. Hopefully they won't find any. Where is the new MBR volume inserted? If it's another USB HD, new handlers will be set up, as with the previous RDB HD. The RDB's handlers will already be gone, so they won't cause any trouble. New MBR volume, in the same physical hotplug slot as the old one (device & unit). The Exec device needs to do the partition rescan on reinsertion, correct? And *not* pass IOHF_MEDIA_CHANGE to any 'stale' handlers until it has verified that those handlers match the original RDB they were opened on, right? They don't see the disk-change, and the insertion of the new volume will create new DOS Devices for each new partition. There is indeed a problem here, in whether we want to Action_Die and reuse device numbers. There are probably advantages to both.

So... What now? Should the ata-0 device rescan for new partitions? Yes. Some other task/handler?

Should the old Dos devices hang around (ACTION_INHIBITed), until we see something that looks like the old RDB again? Not in the case I described above (USB HDs). A more interesting situation would be something like a DVD-RAM drive that accepts partitioned removable media. IMHO the handlers should be sent ACTION_DIE when partitioned media is removed, but the Exec block device would stay. This would probably work for SATA HDs too. Yes, at least those that were inhibited. Here comes the effect of the question about reusing the numbers. If we reused those that had no lock, then reinserting the stick may give them different device numbers.

What if they won't inhibit? But we'd better not pull the stick, because they are busy writing to it. Should we ACTION_DIE the DOS Devices from the old RDB? Yes, since they were created dynamically for a situation that no longer exists. No, because we may want to reinsert that volume. We'll have to weigh that one. Does any one know a reference?

What if they won't die? Then there's a bug in that OS component.