X-Git-Url: http://git.jankratochvil.net/?p=www.jankratochvil.net.git;a=blobdiff_plain;f=project%2Fcaptive%2Fdoc%2FDetails.pm;fp=project%2Fcaptive%2Fdoc%2FDetails.html.pl;h=0a0efd19e6f4ae9d012ec2a125d9311cd8f94671;hp=bb6522ffb1711c701b910c73fb1b7a764487492b;hb=f40e75167a045d189c3027a0b112a20c635d3e48;hpb=ef80d25b0a0307ee222d0a94eaae8abf1e9df31c diff --git a/project/captive/doc/Details.html.pl b/project/captive/doc/Details.pm similarity index 86% rename from project/captive/doc/Details.html.pl rename to project/captive/doc/Details.pm index bb6522f..0a0efd1 100755 --- a/project/captive/doc/Details.html.pl +++ b/project/captive/doc/Details.pm @@ -1,8 +1,6 @@ -#! /usr/bin/perl -# # $Id$ # Captive project doc Details page Perl template. -# Copyright (C) 2003 Jan Kratochvil +# Copyright (C) 2003-2005 Jan Kratochvil # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -25,17 +23,17 @@ our $CVS_ID=q$Id$; use strict; use warnings; -BEGIN{ open F,"Makefile"; our $top_dir=pop @{[split /\s/,(grep /^top_srcdir/,)[0]]}; eval "use lib '$top_dir'"; close F; } use My::Web; -require CGI; -BEGIN { Wuse 'project::captive::doc::Macros'; } +sub handler +{ + BEGIN { Wuse 'project::captive::doc::Macros'; } project::captive::doc::Macros->init( "__PACKAGE__"=>__PACKAGE__, "title"=>'Captive NTFS Developer Documentation: Implementation Details', - "rel_prev"=>'CacheManager.html.pl', - "rel_next"=>'APITypes.html.pl', + "rel_prev"=>'CacheManager.pm', + "rel_next"=>'APITypes.pm', ); @@ -44,13 +42,13 @@ print <<"HERE";

Implementation Details

-

Choice of the Emulation Methods

 +

Choice of the Emulation Methods

The intent of the project was to get reliable read-write access to NTFS partition. There are several possible ways to achieve that:

-

Virtualmachine Running the Original W32 Subsystem

 +

Virtualmachine Running the Original W32 Subsystem

Creating virtual-hardware PC and running the original W32 binaries including their boot-loader etc. Disk device access would be passed as @@ -70,7 +68,7 @@ print <<"HERE"; requirement of fully installed Microsoft Windows NT product.

-

"ntoskrnl.exe" Inside Virtual Address Space

 +

"ntoskrnl.exe" Inside Virtual Address Space

This solution was chosen by the project. Binary filesystem driver and also ntoskrnl.exe binary file are required. @@ -79,12 +77,12 @@ print <<"HERE"; emulation, therefore such instructions must be trapped and emulated/ignored from case to case.

-

Also the initialization code of ntoskrnl.exe is not executed by this project since +

Also the initialization code of ntoskrnl.exe is not executed by this project since it expects to get full PC hardware access privileges and thus some datastructures do not get initialized by it (need to be trapped later at runtime stage). Some of the missing initializations are solved by - @{[ a_href 'APITypes.html.pl#functype_wrap','API functions wrapping' ]}. + @{[ a_href 'APITypes.pm#functype_wrap','API functions wrapping' ]}.

pros: Lightweight, easier to debug.

@@ -92,12 +90,12 @@ print <<"HERE"; ntoskrnl.exe parts, missing documentation needed for the implementation.

-

Filesystem Driver Inside Virtual Address Space

 +

Filesystem Driver Inside Virtual Address Space

-

Unlike @{[ a_href 'Details.html.pl#method_ntoskrnl','previous method' ]} here we do not use +

Unlike @{[ a_href 'Details.pm#method_ntoskrnl','previous method' ]} here we do not use even ntoskrnl.exe as the complete kernel part of - W32 is emulated from the project source - files. cdfs.sys driver was successfuly ran + W32 is emulated from the project source + files. cdfs.sys driver was successfuly ran in this manner in the former versions of this project but the possibility to run without ntoskrnl.exe was dropped since it had no licensing gains (you need the original @@ -112,23 +110,24 @@ print <<"HERE"; ntoskrnl.exe, its missing documentation.

-

API Function Implementation Choices

 +

API Function Implementation Choices

During the initial point of the project development all the API functions were defined as unimplemented, of course. Any call of such unimplemented function is fatal and results in program termination. When we need to implement any required API function we have multiple choices to do so: - @{[ a_href 'APITypes.html.pl#functype_pass','Direct pass to original ntoskrnl.exe' ]}, - @{[ a_href 'APITypes.html.pl#functype_wrap','Wrap of the original ntoskrnl.exe function' ]}, - @{[ a_href 'APITypes.html.pl#functype_native_reactos','Native implementation – $ReactOS' ]}, - @{[ a_href 'APITypes.html.pl#functype_native_wine','Native implementation – $Wine' ]} + @{[ a_href 'APITypes.pm#functype_pass','Direct pass to original ntoskrnl.exe' ]}, + @{[ a_href 'APITypes.pm#functype_wrap','Wrap of the original ntoskrnl.exe function' ]}, + @{[ a_href 'APITypes.pm#functype_native_reactos','Native implementation – $ReactOS' ]}, + @{[ a_href 'APITypes.pm#functype_native_wine','Native implementation – $Wine' ]} or - @{[ a_href 'APITypes.html.pl#functype_native_libcaptive','Native implementation – project specific' ]}. - + @{[ a_href 'APITypes.pm#functype_native_libcaptive','Native implementation – project specific' ]}. + +

-

Sandboxing of W32 Filesystem

 +

Sandboxing of W32 Filesystem

The emulated W32 environment running the original W32 filesystem driver is separated from the rest of UNIX OS. It achieves the following goals:

@@ -166,7 +165,7 @@ print <<"HERE"; peers.

-

"patched" vs. "unpatched" Libraries

 +

"patched" vs. "unpatched" Libraries

Library is called patched if we require loading its original binary code file. Project needs to patch it to be able @@ -176,7 +175,7 @@ print <<"HERE";

Library is called unpatched if no original binary code is needed since all of its functions are completely emulated by - @{[ a_href 'APITypes.html.pl#functype_native','the native implementations' ]} of this project. + @{[ a_href 'APITypes.pm#functype_native','the native implementations' ]} of this project. The typical unpatched representative is hal.dll as it specializes on the hardware dependent code and therefore it must be completely replaced by this project @@ -185,7 +184,7 @@ print <<"HERE"; native implementation of ntoskrnl.exe but it no longer applies.

-

Memory Management

 +

Memory Management

Original Microsoft Windows NT architecture uses two address space areas – user space and kernel space. @@ -209,7 +208,7 @@ print <<"HERE"; moving operations between W32 kernel space and W32 user space memory areas (such as MmSafeCopyToUser()).

-

Unicode Strings and Characters

 +

Unicode Strings and Characters

W32 platform uses 16-bit type wchar_t while $gnulinux uses a 32-bit one. This can be problem during GCC (GNU C Compiler) @@ -249,7 +248,7 @@ print <<"HERE"; -

Supported Binary Formats

+

Supported Binary Formats

The native W32 binary format is identified as PE-32 (Portable Executable 32-bit), such @@ -283,8 +282,8 @@ print <<"HERE"; PE-32 use the appropriate W32 specific @{[ a_href '#calltype','cdecl/stdcall/fastcall call types' ]}, .so must be completely compiled in the standard - UNIX @{[ a_href 'CallType.html.pl#calltype_cdecl','cdecl call type semantics' ]}. - @{[ a_href 'APITypes.html.pl#functype_native','Native function implementations' ]} do not need + UNIX @{[ a_href 'CallType.pm#calltype_cdecl','cdecl call type semantics' ]}. + @{[ a_href 'APITypes.pm#functype_native','Native function implementations' ]} do not need to be explicitely exported by captivesym as they are resolved automatically by the UNIX dynamic system linker. It may be surprising you will have to fix all such missing symbol exports if you @@ -293,7 +292,7 @@ print <<"HERE"; original PE-32 binary file.

-

At Most One Mounted Filesystem

 +

At Most One Mounted Filesystem

The project technically supports only one (exactly one...) mounted filesystem device and only one filesystem driver. There is nothing @@ -303,7 +302,7 @@ print <<"HERE"; itself. It was considered as a more sane way to support multiple W32 mounted disks by completely separately running project instances in a different UNIX processes communicating from their sandboxes via - @{[ a_href 'Details.html.pl#sandbox','CORBA sandbox interface' ]}. This sandboxing + @{[ a_href 'Details.pm#sandbox','CORBA sandbox interface' ]}. This sandboxing feature is not yet deployed although its code is already prepared.

The project also does not support any state cleanup to be able to load @@ -317,18 +316,18 @@ print <<"HERE"; captive_shutdown() the process address space is no longer usable for any further project operations and the process is expected to be terminated in the manner compatible with its driving - @{[ a_href 'Details.html.pl#sandbox','CORBA sandbox interface' ]} control master.

+ @{[ a_href 'Details.pm#sandbox','CORBA sandbox interface' ]} control master.

Each sandbox executing the untrusted W32 binary filesystem driver code is connected through its - @{[ a_href 'Details.html.pl#sandbox','CORBA sandbox interface' ]} at the point of upper + @{[ a_href 'Details.pm#sandbox','CORBA sandbox interface' ]} at the point of upper layer libcaptive-specific filesystem API, at the point of the bottom layer of GIOChannel device access and also for transfers of GLib logging messages/warnings/errors out of the sandbox to the user.

-

Multithreading and Multiple Processors

 +

Multithreading and Multiple Processors

W32 platform stands on its thorough architecture parallelism. It must lock all its objects to maintain coherence in presence of @@ -355,7 +354,7 @@ print <<"HERE"; g_idle_add_full() with g_main_context_iteration() called during KeWaitForSingleObject().

- Since there is a possibility a real W32 parallel threading would +

Since there is a possibility a real W32 parallel threading would be yet needed in the future all the code that would be hit by W32 multithreading capability is marked by TODO:thread comment.

@@ -376,7 +375,7 @@ print <<"HERE"; is done during cleanup of the project's execution by captive_shutdown().

-

Paranoia Checks

 +

Paranoia Checks

A general approach of software projects development is to implement many internal sanity checks during the development stage but to produce the @@ -426,19 +425,19 @@ print <<"HERE"; has to be fixed, of course.

-

STATUS_LOG_FILE_FULL

 +

STATUS_LOG_FILE_FULL

After writing approx. 1MB of data on NTFS test partition NTFS driver returns for any further write requests - STATUS_LOG_FILE_FULL error code. + STATUS_LOG_FILE_FULL error code. Apparently it is caused by the fact this project is - @{[ a_href 'Details.html.pl#synchronous','single-threaded' ]} and it ignores the spawn + @{[ a_href 'Details.pm#synchronous','single-threaded' ]} and it ignores the spawn of parallel journalling thread during ntfs.sys initialization.

Fortunately ntfs.sys will clear its journalling log file during filesystem unmount. This project will therefore - remount the volume if STATUS_LOG_FILE_FULL + remount the volume if STATUS_LOG_FILE_FULL is detected to workaround missing journalling thread.

Similiar behaviour can be seen during write of compressed files — @@ -446,10 +445,10 @@ print <<"HERE"; during the final filesystem unmount.

For these reasons it was mandatory to support - @{[ a_href 'Details.html.pl#parent_connector','transparent volume remounting' ]}.

+ @{[ a_href 'Details.pm#parent_connector','transparent volume remounting' ]}.

-

ParentConnector volume remounter

 +

ParentConnector volume remounter

The sandbox master component of this project has control of restarting its sandbox slaves containing the W32 filesystem. Target goal of @@ -503,4 +502,6 @@ print <<"HERE"; HERE -project::captive::doc::Macros->footer(); +exit; +} +1;