[fpc-devel] Multithreading under DOS (was: [Lazarus] Why development remains constant for msdos?)

[Nikolay Nikolov]

On 09/25/2013 01:45 PM, Mark Morgan Lloyd wrote:
> Nikolay Nikolov wrote:
>> On 09/25/2013 11:26 AM, Michael Schnell wrote:
>>> On 09/24/2013 10:58 AM, Nikolay Nikolov wrote:
>>>>
>>>> When you try to create a thread, your program terminates and writes 
>>>> a message that threading is not supported.
>>>
>>> While this absolutely does make sense, one could think about 
>>> alternatives.
>>>
>>> AFAIK, (at least for some archs) there is a variant of the pthread 
>>> (="POSIX thread") library, that internally does "user-land 
>>> multithreading". IIRC, the original POSIX definition was done with 
>>> exactly this in mind and, regarding Linux, the original Linux 
>>> implementations (aka "Linux Threads")  was not fully compatible with 
>>> POSIX. Only some years ago, the Linux changed it's way of 
>>> Kernel-based thread handling to the POSIX compatible "NPTL" 
>>> implementation.
>>>
>>> Thus it should be possible to link fpc projects to a user-land 
>>> thread enabled version of pthreadlib and allow for working with 
>>> TThread in DOS.
>>
>> I've actually thought about implementing some sort of multithreading 
>> for DOS for a long time. The problems are the following:
>>
>> 1) DOS functions are not reetrant and are thus not safe to call from 
>> different threads. There's an undocumented InDOS flag that indicates 
>> whether a DOS function is safe to call:
>>
>> http://stanislavs.org/helppc/int_21-34.html
>>
>> But the RTL currently doesn't check it before every call and normally 
>> it's only used when writing TSRs.
>
> It's more complex than that: there's undocumented provision in DOS for 
> context switching under certain well-defined conditions, and each 
> context can invoke int 21h irrespective of other contexts' states. 
> That sort of thing was used fairly extensively by- for example- IBM 
> real-time control executives (RIC card etc.) but it wasn't until the 
> 1990s that it leaked to general knowledge see Ralph Brown's list).
Interesting. I didn't know that. That makes it even more feasible to try 
to implement multithreading under real mode DOS.
>
>> 2) In DPMI protected mode applications (such as go32v2), you cannot 
>> modify the return address from within an interrupt handler, which 
>> means you cannot implement your task scheduler as a timer interrupt 
>> handler, because you won't be able to switch to a different context 
>> from there. Doing this would require modifications to the DPMI host 
>> (cwsdpmi.exe) and will not work if another DPMI host is active (such 
>> as when running in a windows dos prompt, etc.)
>>
>> 3) Even if you solve 2), DPMI requires that all code and data touched 
>> from an interrupt handler to be locked, so that it cannot be swapped 
>> out. This is a tedious and error prone task to do from a high level 
>> language such as pascal. You should ensure that your entire 
>> scheduler's code and data are locked. An alternative option is to 
>> switch to a DPMI host, that doesn't support swapping (i.e. 
>> cwsdpr0.exe), but then you lose the virtual memory support (and thus 
>> the ability to run on machines that don't have enough memory).
>>
>> 2) and 3) do not apply to 16-bit MS-DOS.
>>
>> Another option is to implement cooperative multitasking, which would 
>> require each thread to call periodically an yield function. This 
>> solves 1), 2) and 3), but threaded code written for other OSes will 
>> require modifications to run under DOS. However, that's still better 
>> than not running at all.
>
> The DPMI issue sounds... interesting, but if I recall correctly what 
> you do is provide a per-thread entry point analogous to a unix signal. 
> A preemption interrupt transfers control to this, and then a coroutine 
> mechanism- outside the ISR- transfers control to the most deserving 
> thread.
>
> Sorry if that's a bit vague, it's been many years since I played with 
> this. Implementation left as an exercise :-)
Yes, but the problem is, when the ISR occurs, you cannot transfer 
control to this per-thread entry point, because when a hardware 
interrupt occurs, the return address is actually in a hidden ring 0 
stack and you don't have access to it. You can probably do it with a 
known DPMI host, that runs your program in ring 0, such as cwsdpr0.exe.

>
> Whether it's really worth tackling, and whether any implementation can 
> be really reliable, are questions to be considered.
>
It's worth for the fun factor for me at least :) As for reliability, I 
bet you can make it as reliable as Windows 98 was at the time, which is 
not that great, as we all know, but still quite usable.

Oh, and before I forget, later versions of DR-DOS have a version of 
emm386.exe which supports multitasking and has an API that supports 
creating processes and threads (from both real and dpmi protected mode) :

http://www.drdos.net/documentation/multtask/

This should be easy to implement under both i386-go32v2 and i8086-msdos. 
Since the FPC rtl has support for pluggable thread managers, it can be 
put in a unit like cthreads.

Nikolay