project/captive/doc/Reverse.html.pl

   1 #! /usr/bin/perl
   2 #
   3 # $Id$
   4 # Captive project doc Reverse Engineering page Perl template.
   5 # Copyright (C) 2003 Jan Kratochvil <project-www.jankratochvil.net@jankratochvil.net>
   6 #
   7 # This program is free software; you can redistribute it and/or modify
   8 # it under the terms of the GNU General Public License as published by
   9 # the Free Software Foundation; exactly version 2 of June 1991 is required
  10 #
  11 # This program is distributed in the hope that it will be useful,
  12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
  13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  14 # GNU General Public License for more details.
  15 #
  16 # You should have received a copy of the GNU General Public License
  17 # along with this program; if not, write to the Free Software
  18 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
  19
  20
  21 package project::captive::doc::Reverse;
  22 require 5.6.0;  # at least 'use warnings;' but we need some 5.6.0+ modules anyway
  23 our $VERSION=do { my @r=(q$Revision$=~/\d+/g); sprintf "%d.".("%03d"x$#r),@r; };
  24 our $CVS_ID=q$Id$;
  25 use strict;
  26 use warnings;
  27
  28 BEGIN{ open F,"Makefile"; our $top_dir=pop @{[split /\s/,(grep /^top_srcdir/,<F>)[0]]}; eval "use lib '$top_dir'"; close F; }
  29 use My::Web;
  30 require CGI;
  31 BEGIN { Wuse 'project::captive::doc::Macros'; }
  32
  33
  34 My::Web->init(
  35                 "__PACKAGE__"=>__PACKAGE__,
  36                 "title"=>'Captive NTFS Developer Documentation: Reverse Engineering',
  37                 "head_css"=>$doc_Macros_head_css,
  38                 );
  39 My::Web->heading();
  40
  41
  42 print <<"HERE";
  43
  44
  45 <a name="reverse"><h1>Reverse Engineering</h1></a>
  46
  47         <p>This project has no intentions to reverse engineer and document the
  48         filesystem data structures themselves since they are being encapsulated by
  49         the filesystem driver. For these reasons the resources available in
  50         projects such as $LinuxNTFS get out of any possible use. This project goal
  51         is to provide fully compatible API interface to the rest of the W32 system
  52         to persuade the filesystem driver it is running in the native
  53         <span class="productname">Microsoft Windows XP</span> environment.</p>
  54
  55         <p>All the W32 filesystem drivers are running in the W32 kernel address
  56         space and this area of W32 API is not much documented by
  57         <span class="productname">Microsoft</span>. Some API functions are not
  58         documented at all and the others are documented insufficiently for a their
  59         possibly needed reimplementation from scratch. Documentation being
  60         consulted primarily consists of
  61         <span class="productname">@{[ a_href 'http://msdn.microsoft.com/library/default.asp?url=/library/en-us/kmarch/hh/kmarch/kmhdr_6enb.asp','MSDN (Microsoft Developer Network) Kernel-Mode Driver Architecture: Windows DDK' ]}</span>
  62         documentation and also various other 3rd party documentation resources such as
  63         <span class="productname">@{[ a_href 'http://www.osr.com/ntinsider/1996/cacheman.htm',
  64                         'The NT Cache Manager Description' ]}</span>,
  65         <span class="productname">@{[ a_href 'http://www.winntmag.com/Articles/Print.cfm?ArticleID=3864',
  66                         'Learn About NT'."'".'s&nbsp;File-system Cache' ]}</span>,
  67         <span class="productname">@{[ a_href 'http://www.ntfsd.org/archive/',
  68                         'NT File System Developers mailing list archives' ]}</span>
  69         including various
  70         @{[ a_href 'http://www.google.com/search?q=site%3Amicrosoft.com','fulltext searches' ]}
  71         through Internet from case to case.</p>
  72
  73         <p>Sometimes no sufficient documentation was found and some code behaviour
  74         had to be reverse engineered directly from the binaries of
  75         <span class="fname">ntoskrnl.exe</span>,
  76         <span class="fname">cdfs.sys</span>,
  77         <span class="fname">fastfat.sys</span>
  78         and primarily
  79         <span class="fname">ntfs.sys</span>.
  80         Up to now the code was disassembled by
  81         <span class="productname">@{[ a_href 'http://www.simtel.net/pub/pd/29498.html','IDA Freeware' ]}</span>
  82         and by
  83         <span class="productname">dumpbin.exe</span> of
  84         <span class="productname">Microsoft Visual Studio</span>.
  85         <span class="productname">dumpbin.exe</span> is fortunately able to
  86         interpret debug symbols from W32 <span class="fname">.PDB</span>
  87         (Program DataBase) debug information files.</p>
  88
  89         <a name="dumpbin"><h2><span class="productname">dumpbin.exe</span></h2></a>
  90
  91                 <p>You should use the following options for
  92                 <span class="productname">dumpbin.exe</span>:</p>
  93
  94                 <blockquote class="command">
  95                         <p>dumpbin.exe /all /rawdata:none /disasm /pdbpath:verbose FILENAME.SYS</p>
  96                 </blockquote>
  97
  98                 <p>You should see the following line in the output:</p>
  99
 100                 <blockquote class="command">
 101                         <p>PDB file found at '.\\FILENAME.pdb'</p>
 102                 </blockquote>
 103
 104         <a name="WinDbg"><h2><span class="productname">WinDbg</span> Windows NT kernel debugging</h2></a>
 105
 106                 <p><span class="productname">WinDbg</span> is downloadable from:
 107                 @{[ a_href 'http://www.microsoft.com/whdc/ddk/debugging/installx86.mspx' ]}</p>
 108
 109                 <p>This is (the only?) tool able to debug filesystem drivers incl.
 110                 <span class="fname">ntfs.sys</span>. You will need two computers running
 111                 <span class="productname">Microsoft Windows</span> &mdash; one computer will run
 112                 <span class="productname">WinDbg</span> while the other one will be
 113                 frozen in remote Windows NT kernel debug mode. It does not matter which
 114                 <span class="productname">Microsoft Windows</span> version will be run
 115                 on the <span class="productname">WinDbg</span> side. Your goal is to
 116                 successfuly connect <span class="productname">WinDbg</span>:</p>
 117
 118                 @{[ doc_img 'ntdebug-ntfs','<span class="productname">WinDbg</span> Remote NT Kernel NTFS Debugging' ]}
 119
 120                 <p>The most easy way to setup two computers is to use commercial
 121                 <span class="productname">@{[ a_href 'http://www.vmware.com/download/workstation.html','VMware Workstation' ]}</span>
 122                 where you can run two virtual machines simultaneously on single PC
 123                 hardware and you can connect them by a virtual serial port provided by
 124                 <span class="productname">VMware</span>.</p>
 125
 126                 <a name="WinDbg_WinDbg"><h3><span class="productname">WinDbg</span> side setup</h3></a>
 127
 128                         @{[ doc_img 'ntdebug-vmware-windbg',
 129                                         '<span class="productname">VMware</span> virtual serial port'
 130                                                         .' of <span class="productname">WinDbg</span> side' ]}
 131
 132                         <p>You should setup <span class="productname">WinDbg</span> according
 133                         to:</p>
 134
 135                         @{[ doc_img 'ntdebug-windbg-port','Port settings of <span class="productname">WinDbg</span>' ]}
 136                         @{[ doc_img 'ntdebug-windbg-sym','Symbols files location of <span class="productname">WinDbg</span>' ]}
 137
 138                         <span class="constant">Symbols</span> should point to the directory where
 139                         reside files extracted from the symbol archive for your version of
 140                         <span class="productname">Microsoft Windows</span>. In the case of the
 141                         recommended <span class="productname">Microsoft Windows XP Service Pack 1 Checked Build</span>
 142                         you should use:
 143                         @{[ a_href 'http://msdl.microsoft.com/download/symbols/packages/windowsxp/xpsp1sym_x86_chk.exe' ]}</p>
 144
 145                         <blockquote class="command">
 146                                 <p># Rename xpsp1sym_x86_chk.exe contents .pdb files for WinDbg<br />
 147                                 @{[ CGI::escapeHTML(q{for i in *.pdb*;do ext="`echo $i|sed 's/^.*\.pdb\.\(.*\)$/\1/'`";if [ "$i" = "$ext" ];then echo "BAD:$i";break;fi;base="`echo $i|sed 's/\(\.pdb\)\..*$/\1/'`";echo "md $ext";echo "move /-y $i $ext\\$base";done|sort -u|sed 's/$/'`echo -ne '\r'`'/g' >/tmp/rename.bat}) ]}</p>
 148                         </blockquote>
 149
 150                         <p>The resulting <span class="command">rename.bat</span> for
 151                         <span class="command">xpsp1sym_x86_chk.exe</span> can be found at:
 152                         @{[ a_href 'xpsp1sym_x86_chk-rename.bat.zip' ]}</p>
 153
 154                         <p>The resulting directory should contain at least
 155                         <span class="command">sys\\ntfs.pdb</span>
 156                         and
 157                         <span class="command">exe\\ntoskrnl.pdb</span>.</p>
 158
 159                         <p>Your successfuly connected target (after the steps described
 160                         below) should look like:</p>
 161
 162                         @{[ doc_img 'ntdebug-windbg-boot','Successfuly connected <span class="productname">WinDbg</span>' ]}
 163
 164                 <a name="WinDbg_kern"><h3>Setup of the side being kernel-debugged</h3></a>
 165
 166                         @{[ doc_img 'ntdebug-vmware-xpdebug',
 167                                         '<span class="productname">VMware</span> virtual serial port'
 168                                                         .' of the side being kernel-debugged' ]}
 169
 170                         <p>You must use the following options in your
 171                         <span class="command">c:\\boot.init</span> command-line:</p>
 172
 173                         <blockquote class="command">
 174                                 <p>/debug /debugport=COM1 /baudrate=115200</p>
 175                         </blockquote>
 176
 177                         <p>After booting this <span class="command">boot.ini</span>-entry
 178                         should freeze at this point
 179                         (if no <span class="productname">WinDbg</span> is waiting in the other
 180                         virtual machine):</p>
 181
 182                         @{[ doc_img 'ntdebug-wait','Side being kernel-debugged waiting for <span class="productname">WinDbg</span>' ]}
 183
 184
 185 HERE
 186
 187
 188 My::Web->footer();