#! /usr/bin/perl
# 
# $Id$
# Captive project doc Components page Perl template.
# Copyright (C) 2003 Jan Kratochvil <project-www.jankratochvil.net@jankratochvil.net>
# 
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; exactly version 2 of June 1991 is required
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA


package project::captive::doc::Components;
require 5.6.0;	# at least 'use warnings;' but we need some 5.6.0+ modules anyway
our $VERSION=do { my @r=(q$Revision$=~/\d+/g); sprintf "%d.".("%03d"x$#r),@r; };
our $CVS_ID=q$Id$;
use strict;
use warnings;

BEGIN{ open F,"Makefile"; our $top_dir=pop @{[split /\s/,(grep /^top_srcdir/,<F>)[0]]}; eval "use lib '$top_dir'"; close F; }
use My::Web;
require CGI;
BEGIN { Wuse 'project::captive::doc::Macros'; }


project::captive::doc::Macros->init(
		"__PACKAGE__"=>__PACKAGE__,
		"title"=>'Captive NTFS Developer Documentation: Components',
		"rel_prev"=>'Architecture.html.pl',
		"rel_next"=>'Reverse.html.pl',
		);


print <<"HERE";


<h1>Project Components</h1>

	@{[ doc_img 'dia/arch-all','Project Components Architecture' ]}

	<p>Most of the work of this project is located in the single box called
	&quot;<span class="constant">libcaptive</span>&quot; located in the center
	of the scheme. This component implements the core W32 kernel API by
	various methods described in this document.
	The &quot;<span class="constant">libcaptive</span>&quot; box cannot be
	further dissected as it is just an implementation of a&nbsp;set of
	@{[ captive_srcfile 'src/libcaptive/ke/exports.captivesym','API functions' ]}.
	It could be separated to several subsystems such as the
	@{[ a_href '#cache_manager','Cache Manager' ]},
	Memory Manager, Object Manager, Runtime Library, I/O&nbsp;Manager
	etc. but they have no interesting referencing structure.</p>

	<p>As this project is in fact just a&nbsp;filesystem implementation every
	story must begin at the device file and end at the filesystem operations
	interface. The unified suppported interfaces are
	<span class="productname">@{[ a_href 'http://developer.gnome.org/doc/API/2.0/glib/','GLib' ]}</span>
		(the most low level portability, data-types and utility library for Gnome)
	<span class="type">GIOChannel</span> (for the device access) and the custom
	<span class="constant">libcaptive</span> filesystem API. Each of these ends
	can be connected either to some direct interface (such as the
	<span class="constant">captive-cmdline</span> client),
	@{[ a_href 'http://lufs.sourceforge.net/lufs/','Linux Userland File System (LUFS)' ]}
	or as a general $GnomeVFS filter.
	@{[ a_href 'http://lufs.sourceforge.net/lufs/','LUFS' ]} will be used in
	most cases as it offers standard filesystem interface by Linux kernel.
	
	You can also use $GnomeVFS as it offers nice filter interface on
	the UNIX user-privileges level for transparent operation with archives and
	network protocols. This filter interface was used by this project to turn
	the device reference such as <span class="fname">/dev/hda3</span> or <span
	class="fname">/dev/discs/disc0/part3</span> to the fully accessible
	filesystem (pretending being an &quot;archive&quot; in the device
	reference). This device access can be specified by $GnomeVFS URLs such as:
	<span
	class="fname">file:///dev/hda3#captive-fastfat:/autoexec.bat</span></p>

	<span class="constant">captive-bug-replay</span> serves just for debugging
	purposes &mdash; you can 'replay' existing
	<span class="fname">file.captivebug.xml.gz</span> automatically being
	generated during W32 filesystem failure. This bugreport file will contain
	all the touched data blocks of the device used in the moment of the
	failure. <span class="constant">captive-bug-replay</span> will therefore
	emulate internal virtual writable device out of these bugreported data.

	<p>If the passed device reference is requested by the user to be accessed
	either in <span class="dashdash">--ro</span> (read-only) mode or in the
	<span class="dashdash">--rw</span> (full read-write) mode there are no
	further device layers needed. Just in the case of <span
	class="dashdash">--blind</span> mode another layer is involved to emulate
	read-write device on top of the real read-only device by the method of
	non-persistent memory buffering of all the possible write requests.</p>

	<span class="constant">sandbox commit buffer</span> is involved only in the
	case @{[ a_href 'Details.html.pl#sandbox','sandboxing feature' ]} is active. It will
	buffer any writes to the device during the sandbox run to prevent
	filesystem damage if the driver would fail in the meantime. If the
	filesystem gets finally successfully unmounted this sandbox buffer can be
	<a name="safe_flush">safely flushed</a>
	to its underlying physical media. The buffer will be dropped
	in the case of filesystem failure, of course. The filesystem should be
	unmounted from time to time &mdash; it can be transparently unmounted and mounted
	by <span class="command">commit</span> of
	<span class="constant">captive-cmdline</span> custom client. Currently you
	cannot force remounting when using
	@{[ a_href 'http://lufs.sourceforge.net/lufs/','LUFS' ]} interface client
	but it will be remounted after approx each 1MB data written automatically
	due to @{[ a_href '#log_file_full','NTFS log file full' ]}.

	Now we need to transparently
	@{[ captive_srcfile 'src/libcaptive/sandbox/sandbox.idl','connect' ]}
	the device interface of <span class="type">GIOChannel</span> type through
	@{[ a_href 'Details.html.pl#sandbox','CORBA/ORBit' ]} to the sandboxed slave.

	<p>Such device is still only a&nbsp;UNIX style GLib <span
	class="type">GIOChannel</span> type at this point. As we need to supply it
	to the W32 filesystem driver we must convert it to the W32 I/O&nbsp;Device
	with its capability of handling <span class="type">IRP</span>
		(<span class="constant">I/O Request Packet</span>; structure holding the
		request and result data for any W32 filesystem or W32 block device
		operation)
	requests from its upper W32 filesystem driver. Such W32 I/O&nbsp;Device can
	represent either <span class="type">CD-ROM</span> or
	<span class="type">disk</span> device type as different W32 filesystem
	drivers require different media types &mdash; currently only
	<span class="fname">cdfs.sys</span> requires
	<span class="type">CD-ROM</span> type.</p>

	<p>W32 media I/O&nbsp;Device is accessed from the W32 filesystem driver.
	The filesystem driver itself always creates volume object by
	<span class="function">IoCreateStreamFileObject()</span> representing the
	underlying W32 media I/O&nbsp;Device as the object handled by the
	filesystem driver itself. All the client application filesystem requests
	must be first resolved at the filesystem structures level, passed to the
	volume stream object of the same filesystem and then finally passed to the
	W32 media I/O&nbsp;Device (already implemented by this project as an
	interface to <span class="type">GIOChannel</span> noted above).</p>

	<p>The filesystem driver is called by the core W32 kernel implementation of
	<span class="constant">libcaptive</span> in
	@{[ a_href 'Details.html.pl#synchronous','synchronous way' ]} in single-shot manner instead of
	the several reentrancies while waiting for the disk I/O completions as can
	be seen in the original
	<span class="productname">Microsoft Windows NT</span>.
	This single-shot synchronous behaviour is possible since all the needed
	resources (disk blocks etc.) can be always presented as instantly ready as
	their acquirement is solved by @{[ a_href 'hostosnote','Host-OS' ]} outside of
	the W32 emulated @{[ a_href 'guestosnote','Guest-OS' ]} environment.
	For several cases needed only by <span class="fname">ntfs.sys</span> 
	there had to be supported asynchronous access &mdash; parallel execution
	is emulated by GLib <span class="function">g_idle_add_full()</span>
	with <span class="function">g_main_context_iteration()</span> called during
	<span class="function">KeWaitForSingleObject()</span>.</p>

	<p><span class="constant">libcaptive</span> offers the W32 kernel
	filesystem API to the upper layers. This is still not the API the common
	W32 applications are used to as they use W32 libraries which in turn pass
	the call to W32 kernel.  For example
	<span class="function">CreateFileA()</span> is being implemented by several
	libraries such as <span class="fname">user32.dll</span> as a relay
	interface for the kernel function
	<span class="function">IoCreateFile()</span> implemented by this
	project's&nbsp;<span class="constant">libcaptive</span> W32 kernel
	emulation component.</p>

	<p>As it would be very inconvenient to use the legacy, bloated and UNIX
	style unfriendly W32 kernel filesystem API this project offers its own
	@{[ a_href '#client_interface','custom filesystem API interface' ]} inspired by
	the $GnomeVFS client interface adapted to the specifics of W32 kernel API.
	This interface is supposed to be easily utilized by
	<a href="#client_interface_customapp">a&nbsp;custom application accessing
	the W32 filesystem driver</a>.</p>

	<p>@{[ a_href 'Details.html.pl#sandbox','CORBA/ORBit' ]} hits us again &ndash; we need to
	@{[ captive_srcfile 'src/libcaptive/sandbox/sandbox.idl','translate' ]}
	the @{[ a_href '#client_interface','custom filesystem API interface' ]}
	out of the sandboxed slave to the UNIX space.</p>

	<p><span class="constant">captive sandbox master</span> provides the
	functionality of covering any possible sandboxed slave restarts and its
	communication. It is also capable of
	<a name="demultiplexing_master">demultiplexing single API operations</a>
	to multiple its connected sandbox slaves in transparent way
	as each of them handles
	@{[ a_href 'Details.html.pl#mounted_one','just one filesystem device' ]}.</p>

	<p>The rest of the story is not much special for this project since this is
	a common UNIX problem how to offer user space implemented UNIX filesystem
	as a generic system filesystem (as those are usually implemented only as
	the components od UNIX kernel).</p>

	<p>The filesystem service can be offered in several ways:</p>

	<dl>
		<dt>Custom client</dt>
		<dd>
			<p>One possibility would be to write
			<a name="client_interface_customapp">a custom client application</a>
			for this project such as file manager or a&nbsp;shell. Although it
			would implement the most appropriate user interface to the set of
			functions offered by this project (and W32 filesystem API) it has the
			disadvantage of special client software. Appropriate client is provided
			by this project as:
			<span class="fname">src/client/cmdline/cmdline-captive</span></p>
		</dd>

		<dt>@{[ a_href 'http://lufs.sourceforge.net/lufs/','Linux Userland File System (LUFS)' ]}</dt>
		<dd>
			<p>The most usable interface is the
			@{[ a_href 'http://lufs.sourceforge.net/lufs/','LUFS' ]} client
			by <span class="constant">liblufs-captivefs</span>.
			As @{[ a_href 'http://lufs.sourceforge.net/lufs/','LUFS' ]}
			already assigns separate process for each filesystem mount the
			@{[ a_href '#demultiplexing_master','demultiplexing feature' ]}
			is not utilized in this case.</p>

			<p>@{[ a_href 'http://lufs.sourceforge.net/lufs/','LUFS' ]}
			needs multiple operating threads (each UNIX kernel operation needs
			one free lufsd slot/thread to not to fail immediately).
			As <span class="constant">libcaptive</span> is
			@{[ a_href 'Details.html.pl#synchronous','single-threaded' ]} all the operations
			get always synchronized by
			<span class="constant">liblufs-captivefs</span>
			before their pass over to <span class="constant">libcaptive</span>.</p>
		</dd>

		<dt>@{[ a_href '#offered_gnomevfs','Gnome-VFS' ]}</dt>
		<dd>
			<p>This client allowing its filesystem access even without any
			involvement of UNIX kernel from any $GnomeVFS aware client application
			(such as <span class="fname">gnome-vfs/tests/test-shell</span>).
			This @{[ a_href '#offered_gnomevfs','Gnome-VFS interface' ]} connects the
			data flow of this project in two points &mdash; both as the lowest layer
			device image source and also as the upper layer for the filesystem
			operation requests.</p>
		</dd>
	</dl>

	<p>Unimplemented and deprecated methods for providing filesystem
	service:</p>

	<dl>
		<dt>W32 filesystem in UNIX OS kernel</dt>
		<dd>
			<p>The real UNIX OS filesystem implementation must be completely
			implemented inside the hosting OS kernel. This requires special coding
			methods with limited availability of coding features and libraries.
			Also it would give the full system control to the untrusted W32
			filesystem driver code with possibly fatal consequences of yet
			unhandled W32 emulation code paths. It would benefit from the best
			execution performance but this solution was never considered a real
			possibility.</p>
		</dd>

		<dt>Custom NFS server</dt>
		<dd>
			<p>The common approach
			<a name="offered_NFS">of filesystem implementations</a>
			outside UNIX OS kernel were custom NFS servers usually running on the
			same machine as the NFS-connected client as such NFS server is usually
			an ordinary UNIX user space process. It would be possible to implement
			this project as a&nbsp;custom NFS server but the NFS protocol itself
			has a&nbsp;lot of fundamental flaws and complicated code for backward
			compatibility.</p>
		</dd>
	</dl>


HERE


project::captive::doc::Macros->footer();
