Telegard 3.0 Development Kit (v3.01)
June 9th, 1996
Copyright 1995,1996 by Tim Strike.
All rights reserved.
INTRODUCTION I
_____________________________________________________________________
PREFACE
This documentation was designed to help utility developers find
information regarding Telegard data files and formats with
relative ease and speed.
This kit is a revision to the 3.0 kit as released, and documents
the changes from gamma-1 through the final 3.0 release.
Before developing utilities for Telegard, it would be appreciated
if this document was thoroughly read. This documentation contains
several discussion sections which outline methods, procedures and
data file formats. The discussion section explains all of the
data files that Telegard maintains, and how to access the data
and what must be done to properly access the information therein.
It is imperative that important items in this document are
followed when developing utilities for use with Telegard. By
following these specifications and helpful hints, you reduce the
chance of producing a poorly written/working utility.
Telegard 3.0 brought about a major revision in the way that many
of the data files are used. You will find that many of the files
with the same name no longer use the same data format; please
ensure that even when UPDATING utilities that you look at the
information which is provided for each file type (if they are not
listed under major headings in the DEVELOPMENT DISCUSSION section,
they will be listed under the heading OTHER FILES).
After the discussion section follows a section on distribution.
This section outlines the best ways to document your work, the
best ways to ensure accurate distribution and even how to access
the Telegard File Distribution Network, which will distribute your
files across North America and overseas and give you the widest
distribution for nothing more then a simple phone call to one of
the development sites (provided that your distribution package
follows the simple guidelines that are set forth in this section).
We urge the development of quality utilities for Telegard 3.0. We
also urge you to look over this document in it's entirety so that
you will be able to accomplish your goal of writing a utility
which works properly and efficiently with Telegard.
CONTACT INFORMATION
The best way to contact the Telegard development team is through
our support conference on Fidonet, TG_SUPPORT. In this conference
we will answer all questions relating to Telegard and Telegard
third-party utility development.
The next best way is to contact us through Fidonet/Internet
netmail.
The author of this document can be reached via the following
addresses:
Fidonet 1:259/423
Internet ts@gryn.org
Telegard BBS Development
11-3360 Council Ring Rd
Mississauga, ON
L5L 2E4
Please feel free to contact us with regard to Telegard utility
development -- questions relating to data file format, access or
any other questions are welcomed.
INCLUDED FILES
Several files have been included in this package. If any of these
files are missing, please obtain a new development package from
one of our development sites.
DEVEL301.DOC Telegard 3.0 Development Documentation (3.01)
README.TG 'Read-Me' Information
TELEGARD.H C/C++ structure file
TELEGARD.INC Turbo/Borland Pascal structure file
TEXT_ADF.TXT Text File Description - Archive Definitions
TEXT_MDF.TXT Text File Description - Modem Definitions
TEXT_PDF.TXT Text File Description - Protocol Definitions
FILE_ID.DIZ Automated package description
PACKING.LST Package file listing
DEVELOPMENT DISCUSSION II
_____________________________________________________________________
SOURCE CODE EXAMPLES
Secondary packages will be released at a later date which provide
source code examples to many of the topics discussed in this
document. If at the time of this reading the packages are not yet
available, please feel free to contact us with regards to any
questions.
The reason that the source code examples are not provided with
this release is relatively simple -- source code examples can be
improved and extended, and as a result, we've chosen to release
Telegard coding 'magazine' type packages, where new examples will
be provided and where common questions can be answered on a
regular basis.
The first package should be released shortly, and will contain
preliminary code examples in both Pascal and C/C++. If you have
problems you would like to see source code answers for, or have
questions you would like to ask regarding programming with
Telegard, please ensure that you contact us so that we can look
them over for the next source code package.
The source code packages will likely take the filenames
TGDEV@nn.ZIP where @ will be an alphanumeric volume number (A=1,
B=2, etc) and nn is the release in that volume. Announcements
will be made when these packages are released.
TELEGARD 3.0
Telegard 3.0 was developed in Borland Pascal 7.0 -- with portions
compiled in C and Assembly.
All the Telegard 3.0 internal data structures use standard pascal
methods and constructors, and this should be noted by utility
developers who will use other languages. Structures which are
external to Telegard 3.0 (i.e. data files which other programs
create and Telegard just uses, things like JAM, Squish, etc.) use
either C or Pascal methods and constructors; ensure that you
determine which is which and make the necessary adjustments in
your programs.
DATE FORMATS
Telegard 3.0 uses two different types of date formats; the first
is a simple string format and the second is a Unix timestamp.
STRING DATE FORMAT -- -- -- -- -- -- -- -- --
The string dates are always in the same format (MM/DD/YY) and are
converted to other formats on the fly. This date format is stored
in a string of length 8.
UNIX DATE FORMAT -- -- -- -- -- -- -- -- -- --
The second date format is a Unix timestamp. Effectively this is
the number of seconds since midnight, January 1st, 1970. Several
compilers and implementations use different algorithms to
calculate the date format. Telegard uses the local system time
(i.e. the time on the machine) and calculates the number of
seconds since midnight, January 1st, 1970, including the extra
days during all leap years (which some implementations ignore).
No adjustments are made for timezone, and no conversion is done
to Greenwich Mean Time (GMT).
CONFIGURATION FILES
Data file CONFIG.TG -- -- -- -- -- -- -- -- --
The main configuration is stored in CONFIG.TG in the main BBS
directory. This file contains all paths and all necessary
configuration items that pertain to operation and selection of the
BBS' features. The CONFIG.TG file can be opened and modified;
please note that modifications to the file will not immediately be
loaded into memory in a multinode environment. If another task
overwrites the file contents, another node could still overwrite
and replace those contents at another time.
The hiddenPW field should *never* be updated by an external
utility, as the consequences of doing this could be very dire.
Data file SYSTEM.DAT -- -- -- -- -- -- -- -- --
Telegard 3.0 introduces a secondary configuration file, which
contains all the SYSTEM variables, and thus, is called SYSTEM.DAT
(stored in the /DATA/ directory). The SYSTEM.DAT file is very
sensitive to changes, and thus, the following procedure must be
followed (in single node or multinode environments):
(1) Open file
(2) Obtain a lock on the entire files contents - if
lock can not be obtained, cycle for waiting period
abort w/error if still unable
(3) Read the current data record into memory
(4) Update record (do *NOT* sit idle waiting for input
or other processes)
(5) Write the new information to the file
(6) Unlock file contents, close file
The information in this record must be current, and always must be
accessed by only one source at a time, which is why the above
procedure must be followed.
SEARCHING INDEXES
Several of the data files used in Telegard 3.0 are indexed to make
searching for data quicker and easier to accomplish. The index
files, marked with TG_INDEX in the structures, contain different
information but are all built and maintained the same way.
The purpose of the index files (*.IDX) is to ensure quick access
to the information. The index file design used in Telegard
facilitates not only the searching of existing records, but also
the addition of new records.
The records in the index files all contain some data [key] and
a number location [offset]. The [key] is used to find the
correct record, and the [offset] can be used with another file
to directly load the necessary information.
The index files (*.IDX) have two distinct sections; the first is a
sorted database, and the second part is an unsorted database (used
primarily for additions to the index). A binary search is used
for the first section, and a sequential search for the second
section. The binary search uses the sorted characteristic to
chop the search area in half each time; this results in a very
quick search of the sorted section of the database. If the [key]
can not be found there, the sequential search will check through
the small unsorted portion. If the [key] is not found there, the
[key] does not exist. The important thing to note about this
indexing method is that the sorted database will be significantly
large compared to the unsorted portion. Frequent updates of the
database using the indexing utilities provided with Telegard will
ensure that the fastest searching time is always maintained.
When the index files are first built, all valid records will be
added to the file and sorted. When new records are created, the
records will be added to the end of the database and remain there
until index maintenance is completed.
The first record in the index files contains the record offset
of the first unsorted record. This is important to note, since
the binary search should ONLY be run on the database starting at
record #1 and ending at the last sorted record. The sequential
search should start at the first unsorted record and end at the
last record in the index file.
Record 0 Unsorted index record [offset=NS]
Record 1 First sorted record, if NS <> 1
Record NS-1 Last sorted record
Record NS First unsorted record -- could be <EOF>
to <EOF> Last unsorted record
To search for a [key] the following procedure should be followed:
1. Read record #0, store [offset]=NS
2. Binary search on record #1 through record NS-1.
If [key] found, skip to FOUND.
3. Sequential search of record NS through <EOF>.
If [key] found, skip to FOUND.
4. [key] does not exist, skip to END.
FOUND. If [offset] = -1, [key] does not exist, skip to END.
Otherwise, [key] exists, use [offset] to find data.
END.
The indexing methods used in Telegard 3.0 provide fast and
efficient loading and manipulation of the data. There are other
ways to search for information, but if the index files exist for
the information you are looking for, they are the best way to find
the information.
ADDING INDEX RECORDS -- -- -- -- -- -- -- -- --
Search the index to see if a record already exists for that [key].
If it does exist, there are two possibilities (a) this is an
adjusting record, in which case adjust the [offset] field and
rewrite the record or (b) this [key] already exists and does not
need to be added, or can not be added. If is does not exist,
simply create the necessary record and then write it to the end
of the index file (simple, eh?).
REMOVING INDEX RECORDS -- -- -- -- -- -- -- --
Search the index for the necessary [key] - if a record already
exists, set the [offset] field to -1 and rewrite the record. If
the record doesn't exist, it can't be deleted now can it (smile).
USERS
Telegard 3.0 no longer uses USER NUMBERS to find records in the
data files. Record #2, therefore, is not User #2 like older
versions of Telegard had implemented.
USER-ID -- -- -- -- -- -- -- -- -- -- --
Since user numbers were no longer tied to user records, a user-id
was created to give each user a 'id-tag' that was not associated
with user/real names (since those change frequently). The user-id
number is generated when the user first logs on and is ordered
sequentially from 1 (i.e. the first available user-id after 0 is
assigned to the new user). This user-id field is only used to
reference the user' records in other files. It is *not*
associated with the users data file position; i.e. user-id #2 is
not necessarily found in record position #2.
PASSWORD TEXT/CRC FIELDS -- -- -- -- -- -- -- --
The pwtext field *may* be empty. Please observe the following
when updating passwords:
1. All password input is UPPERCASE
2. If ConfigRec.HiddenPW = TRUE, then only the CRC32 of the
password is stored in the user record (userrec.crcpw).
3. If ConfigRec.HiddenPW = FALSE, both the CRC32 and the actual
password are stored in the user record.
*4. When checking input passwords against the file, only the CRC32
value should be tested (the pwtext may or may not be correct).
If ConfigRec.HiddenPW = FALSE, and the pwtext field is EMPTY,
then after a correct password the field will be automatically
updated. If ConfigRec.HiddenPW = TRUE, the pwtext will *not* be
updated (this is obvious).
*5. Passwords should *only* be displayed (pwtext) if the CRC32 of
the pwtext field *matches* the pwcrc field. Otherwise, the
pwtext field is likely incorrect. When changing passwords, the
user should be prompted only, not shown their previous password.
*6. The CRC32 calculation used by Telegard is a standard CRC32
calculation. The initial value is 0xFFFFFFFF (FFFFFFFFh)
and only the contents of the password are uses (i.e. start at
position 1, not 0 (the length byte). Remember, the passwords
are in uppercase.
Data file USERS.DAT -- -- -- -- -- -- -- -- --
The USERS.DAT data file has no specific order, except that the
SysOp Record must be record #0. The data file may be sorted in
any manner that is deemed important, except that record #0 should
never be moved; that is the only restriction to the order of the
file.
Data file USERS.IDX [TG_INDEX] -- -- -- -- -- -- --
To speed up data searches a USERS.IDX file has been created
(replacing NAMES.LST from older versions of Telegard). The index
file contains both user aliases and real names (if different) so
that searches can be completed using either one. The offset
provided in the index record is the record position offset in
USERS.DAT.
Data file USERID.IDX [TG_INDEX] -- -- -- -- -- --
To speed up associative data searches, a USERID.IDX file has been
created. The index file contains the userid's of all active users
so that searches can be completed by finding the matching id. The
offset provided in the index record is the record position offset
in USERS.DAT.
SEARCHING FOR USERS -- -- -- -- -- -- -- -- --
Use the index files to quickly search for users (user name/alias
or id) or use a sequential search on USERS.DAT to search for other
fields.
DELETING USERS -- -- -- -- -- -- -- -- -- --
Deleting users from the database is a relatively simple task, but
has many associated items which must be properly executed for the
data files to be tidy and efficient.
(1) Remove user records from *.IDX files (see section on
TELEGARD INDEXES - REMOVING INDEX RECORDS )
(2) Remove all email (base 0) to user
(3) Remove all shortmsgs (SHORTMSG.DAT) to user
(4) Adjust voting records for user (VOTING.DAT - adjust totals)
(5) Reset all lastread pointers to ZERO
(6) Remove macro record of user (null MACRO.DAT.UserId, set
macro pointer to -1)
(7) Set USERID field to ZERO
(8) Set udeleted in user flags
Following those instructions will lead to the most compact data
files and ensure that data records are not accidentally
cross-linked.
PACKING USERS -- -- -- -- -- -- -- -- -- --
All deleted users can be removed from the data file by simply
removing their data record (provided that the users were properly
deleted as above). All this entails is that the users record is
overwritten with an active record, or a new data file is created
with only the active user records).
SORTING USERS -- -- -- -- -- -- -- -- -- --
Since the dependency on user numbers no longer exists, user
records can be sorted in any manner with any method. The only
restriction (as noted above) is that Record #0 must remain as
Record #0 at all times (it should *not* be sorted).
MESSAGES
Telegard 3.0 no longer uses proprietary message formats, but
instead uses the standard formats SQUISH and JAM. These two
formats provide standard access to the message bases, are
considerably more adaptable and provide similar performance to
the old Telegard formats without losing compatibility.
Data file MAREAS.DAT -- -- -- -- -- -- -- -- --
The MAREAS.DAT data file stores all the datafile definitions for
Telegard message bases. There are only a few areas of note in the
record format which will be outlined shortly. The message bases
are stored in numerical order (i.e. base #1 is record #1) -- this
number will correspond to the message base number within Telegard
unless the base compression tables are on (this will have to be
handled differently by other utilities).
* If the FileName is blank, the message base is not active
and should not be processed.
* The *.MSI, JAM/Squish files will be stored in the MsgPath
field -- if they are not there, create them there.
* The QwkIndex field should be a permanent number which
should not be changed except for extreme repair procedures
or for controlled maintenance purposes; the QwkIndex field
is best changed by the indexing utility provided with
Telegard.
* When changing a FileName the MAREAS.IDX file should be
updated; the old record should be deleted and the new
record should be created (see section on TELEGARD INDEXES -
ADDING AND REMOVING INDEX RECORDS).
* Similarly, if and when changing a QwkIndex number, the
QWK.IDX file must be updated in the same fashion.
Data file MAREAS.IDX [TG_INDEX] -- -- -- -- -- --
The MAREAS.IDX file contains the base filenames and file offsets
into the file MAREAS.DAT for those bases. This file can be used
to quickly locate a specific base by filename.
Data file QWK.IDX [TG_INDEX] -- -- -- -- -- -- --
The QWK.IDX file contains the QWK base numbers (QwkIndex) and the
file offsets into the file MAREAS.DAT for those bases. This file
can be used to quickly locate a specific base by QwkIndex number.
Data file *.MSI -- -- -- -- -- -- -- -- -- --
The *.MSI data files contain the base selection tables for all
users. To find the scan record for any user, the offset in the
*.MSI file is userrec.userid-1 (so if the userid = 2, the offset =
1 and therefore the record position = 1).
JAM FORMAT -- -- -- -- -- -- -- -- -- -- --
The JAM format uses an expandable record format which essentially
allows it to grow as needed for application purposes. The purpose
of this section is not to outline how JAM works (the JAM API
released by the authors of JAM provides code examples and a text
development document) but rather to just quickly outline the
format and some assumptions used with Telegard 3.0.
The *.JHR file contains most of the information that is required
for messages. The first record contains information about the
base that is being looked at. The records afterward are variable
in length because of the numerous subfields that may or may not
exist for each message:
jaminforec
jamhdrrec[1]
subfield[1]
subfield character buffer
:
subfield[n]
subfield character buffer
jamhdrrec[2]
:
jamhdrrec[n]
The subfield codes used by Telegard 3.0 match all the standard JAM
subfield codes.
** NOTE: For bases where anonymous, anyname, etc postings are
used, the sender/receiver names will be "anonymous", "anyname",
etc. The FTS-kludge "^aREALNAME: <name>" will be used to store
the posting users name (which will depend on the realname/alias
toggled on the base). This kludge uses the standard subfield-id
of 2000.
JAM stores its own lastread pointers in the *.JLR files. To find
the correct lastread record for any user, a sequential search for
the matching user-id should be done.
SQUISH FORMAT -- -- -- -- -- -- -- -- -- --
The Squish format is another dynamic message base, which allocates
itself in data chunks (frames) and links itself on the fly. The
Squish bases automatically attempt to pack themselves for all
newly posted or incoming messages, which alleviates the job of
running a packer each night for the Squish message format (it is
however a good idea to run the packer on a frequent basis to
remove the unused frames and reduce disk usage (even at the lowest
level)).
The Squish format is outlined well by it's author in the SQUISH
API package, which includes a comprehensive discussion of the
Squish format and methods for reading, writing and indexing the
message areas.
The Squish format uses it's own lastread pointers, stored in the
*.SQL files. To find the lastread record for any user, the offset
in the *.SQL file is userrec.userid-1 (so if the userid = 2, the
offset = 1 and therefore the record position = 1).
QWK OFFLINE MAIL FORMAT -- -- -- -- -- -- -- --
Telegard 3.0 implements the QWK mail format as per the QWK
specifications (covered well in the document package dealing with
the QWK layout, normally distributed with filenames similar to
QWKLAYnn.ZIP, where nn is the revision number (the latest at the
time of this release was nn=17).
Data file *.QPT -- -- -- -- -- -- -- -- -- --
Telegard 3.0 also adds it's own pointer file, *.QPT which stores
the lastread pointers for the requested message bases. This file
only contains the QWK base number and the lastread number, and is
sent with the QWK offline mail package to the user.
FILES
Telegard 3.0 uses a proprietary file base format, since there are
few standard filebase formats which are suitable and efficient.
Data file FAREAS.DAT -- -- -- -- -- -- -- -- --
The FAREAS.DAT data file stores all the datafile definitions for
Telegard file bases. There are only a few areas of note in the
record format which will be outlined shortly. The file bases are
stored in numerical order (i.e. base #1 is record #1) -- this
number will correspond to the file base number within Telegard
unless the base compression tables are on (this will have to be
handled differently by other utilities).
* If the FileName is blank, the file base is not active
and should not be processed.
* The *.FSI, *.FB and *.FBE files will be stored in the
FilePath field -- if they are not there, create them there.
* Files are actually stored in the path pointed to be
FAreaRec.Path. Offline files are obviously not here.
Check CD-ROM status to check access for writing to this
path (read-only toggle means no writing!).
* Telegard keeps track of CD-ROM titles that are in the
drive at the current time - if the FAreaRec.CdLabel field
contains a value and the area is toggled as CD-ROM Telegard
will check the drive for the correct CD-ROM before trying
to download/access.
* When changing a FileName the FAREAS.IDX file should be
updated; the old record should be deleted and the new
record should be created (see section on TELEGARD INDEXES -
ADDING AND REMOVING INDEX RECORDS).
Data file FAREAS.IDX [TG_INDEX] -- -- -- -- -- --
The FAREAS.IDX file contains the base filenames and file offsets
into the file FAREAS.DAT for those bases. This file can be used
to quickly locate a specific base by filename.
Data file *.FSI -- -- -- -- -- -- -- -- -- --
The *.FSI data files contain the base selection tables for all
users. To find the scan record for any user, the offset in the
*.FSI file is userrec.userid-1 (so if the userid = 2, the offset =
1 and therefore the record position = 1).
Data file *.FB -- -- -- -- -- -- -- -- -- --
The *.FB data files contain the actual file records for the files
that are available and currently in that file base. File records
are stored with a variable length description (which will be
touched on later).
Unlike previous versions of Telegard, record #0 is now used for
file records. Deleted records are completely removed from the
file instead of sitting at the end of the file (like previous
versions as well).
The fields in the file records are not updated on the fly; if a
file is moved offline external to Telegard, then Telegard will not
know this unless the file has its status changed.
Data file *.FBE -- -- -- -- -- -- -- -- -- --
The *.FBE data files contain the extended file descriptions (the
variable length portion). Records with blank filenames are unused
records and can be reused by other utilities. If the entire
record (5 lines) is not needed for a description, null the other
description entries (i.e. fill with #0) - or at least ensure that
their string length is 0.
Variable Length Descriptions -- -- -- -- -- -- --
As mentioned before, Telegard stores the file records with
variable length descriptions. Although the system requires a
linked list, when the list is maintained properly, Telegard can
handle descriptions of up to 26 lines internally (and essentially,
unlimited amounts externally).
The descriptions are organized as follows:
File (FBREC)
Description { first line of description }
Extended { pointer to first extended }
{ description, -1=none }
Extended (FBEXTREC)
Filename { used to verify correct link }
Desc { next 5 lines of description }
{ if null'' line, ignore rest }
{ blank lines should contain }
{ at least one space }
Nextlink { pointer to next extended }
{ description, -1=none }
Description*
File.Extended -> Desc*
Extended.NextLink -> Desc*
Extended.Nextlink -> -1
The description lines (*) can then be composed (in this example)
of up to 11 lines (1+5+5). The description ends when the nextlink
is -1.
Data file FILES.IDX [TG_INDEX] -- -- -- -- -- -- --
The data file FILES.IDX contains a listing of all the filenames
that are currently available on the system. It can be used to
check for duplicate files, to find files quickly (i.e. Request
Processors) or many other useful tasks. The file *must* be
maintained properly. If a file by the same filename already
exists in the index, the utility should determine which is the
correct entry and use that (for this reason, two files can not
have the same filename -- it's a good thing to follow, regardless
of whether or not they are in different file directories).
IMPORTANT NOTE: As of Telegard 3.0, the fileidxrec.filerecord
field is not being used - to find the file, just open the filebase
and perform a sequential search of the records. The file offset
field will be implemented at a later date.
ADDING FILES -- -- -- -- -- -- -- -- -- --
When adding a file to a directory, the index should first be
checked for an existing filename. If the file already exists, the
utility should either (a) replace the old file with the new file,
removing the old file if necessary from other areas or (b) abort
adding the new file to the database.
To add a file, simply write it's record into the correct *.FB
file. If the file has an extended description, update the
FBRec.Extended pointer and link the description into the *.FBE
file. Then add the file into the FILES.IDX (see information on
Telegard Indexes - Adding Index Records).
DELETING FILES -- -- -- -- -- -- -- -- -- --
A little more care should be taken when deleting files. First,
if the file has extended description records, those description
records must have their filenames set to null (empty string) --
just follow the link through and set all corresponding filenames.
After this has been complete, the file should be removed from the
*.FB file (rewrite all records after that record to their current
position-1, truncate the last record from the file -- this ensures
that the order of the *.FB file is maintained). Finally, the
record must be removed from the FILES.IDX file (see information on
Telegard Indexes - Removing Index Records). The file should be
removed from the filepath if (a) necessary and (b) if possible.
Data File PROTOCOL.DAT and Transfer Protocols -- -- -- --
Telegard maintains the transfer protocols in a file called
PROTOCOL.DAT. Internally to Telegard, the protocols are referred
to as A-Z (i.e. 26 protocols can be stored in Telegard - one
record per protocol type). Unlike previous versions of Telegard,
single, batch and resume protocols no longer require separate
records. Record #0 refers to Protocol A, and record #25 refers to
Protocol Z. All the protocol records are created in the file at
once, even if they are not used.
If the ProtRec.Batch field is toggled TRUE, then the files will be
compiled into a filelist and this filelist will be sent as the
protocol filename (~PF). Most protocols require a special flag to
list this file as a 'list' of files - this should be stored in the
ProtRec.flist field. If the ProtRec.Bidirec field is toggled
TRUE, Telegard will search the temporary incoming directory for
files after the download.
If ProtRec.Desc field is null, the protocol has not been defined.
Data file *.QQQ, *.QQD and -- -- -- -- -- -- -- --
Upload and Download Queues
The upload/download queues are stored in the temporary directory
as the files *.QQQ and *.QQD. The upload/download queues store
information about flagged files that are pending upload or
download by the user. Information may be added/removed from these
queues during a shell since the queues will be re-processed after
the shell.
The maximum number of queued files is maxqueue, any entries after
the maxqueue number will not be read by Telegard.
The extended descriptions queues (*.QQD) use the same format as
the filearea extended description files (*.FBE).
Data file TESTINFO.DAT -- -- -- -- -- -- -- --
TESTINFO.DAT is a file that is created by virus scanners and
upload processors. It includes information on file conversions,
descriptions, testing information, etc. Telegard will read this
file (if toggled on) to process conversions and get information
regarding the file testing that has just occurred (after the virus
scanner has been run on the file).
Records which are not scanned for a period of time can be removed
from the TESTINFO.DAT file - this indicates that errors have
occurred with the importation of these file records.
THDPRO V11 and subsequent versions will produce this file. At the
time of this writing, this is the only known product to use the
format. Telegard will only work with THDPRO V11, since prior
versions do not include the TG3.0 product code.
OTHER FILES
If the data file is not listed below, then no extra information is
required to access the data file; the record format is straight
forward, and the data is not organized in any varying manner
(usually there is only one record).
Data file *.MNU -- -- -- -- -- -- -- -- -- --
The menu files are stored in a number of different paths; the
first being the default *.MNU path, stored in CONFIG.TG. The
other paths are language specific paths (for more information see
section LANGUAGE CONSIDERATIONS).
The menu files have a base header (MenuRec) and are followed by up
to MaxMenuCommands of the menu command records (CommandRec). The
*.MNU files must be opened in non-record specific file formats
(i.e. for those Turbo Pascal people, define the type as FILE and
use BLOCKREAD/BLOCKWRITE to access the records).
Data file *.BBS -- -- -- -- -- -- -- -- -- --
The BBSlisting files are stored in the ConfigRec.DataPath and
contain the BBS listing entries. To add an entry, add to the end
of the file. To remove an entry, physically remove the record.
If updating the entry, be sure to touch the BBSListRec.LastEdit
field with the current date. MCI codes should be removed from all
BBSlist entries.
Data file ARCHIVE.DAT -- -- -- -- -- -- -- -- --
Telegard maintains the archivers in a file called ARCHIVE.DAT.
Internally to Telegard, the archivers are referred to as 1-MaxArcs
(i.e. MaxArcs archivers be stored in Telegard - one record per
archive type). Record #0 refers to Archiver #1, Record #1 to
Archiver #2, etc. All the archive records are created in the file
at once, even if they are not used.
If ArchiveRec.Extension field is null, the archiver has not been
defined.
Data file EVENTS.DAT -- -- -- -- -- -- -- -- --
The events that Telegard maintains are stored in the file
EVENTS.DAT. This file contains an array of all the events:
Event(#1) - Event(MaxEvents) and includes a reserved record
Event(#0) which is used for processing external events.
If the event is forced, users will be kicked off when the event
time occurs. If the event is not forced, users will be allowed to
stay on past the duration of the event and the event will be run
afterwards.
If Telegard is loaded from an external program (Mailer, external
call answer door, etc.) the Telegard events will still be adhered
to. If the External Event command line is used (-X[n]) then
Telegard will also include this event time in its event
calculations.
Data file GROUPS.DAT -- -- -- -- -- -- -- -- --
The groups that Telegard maintains are stored in the file
GROUPS.DAT. This file contains an array of all the groups: Group
@ is record #0, Group A is record #1, etc.
If GroupsRec.Name field is null, the group has not been defined.
Data file HISTORY.DAT -- -- -- -- -- -- -- -- --
Telegard stores the call information in a file called HISTORY.DAT.
Unlike previous versions of Telegard in which this file was stored
in reverse order, Telegard 3.0 stores this file in ascending order
- i.e. the first record is the oldest record, the last record is
the newest record.
To find the current days entry, check the last record in the file.
If the dates do not match up, the daily turnover event has not
been run; a new record can be created for the new day (this will
however result in Telegard not processing it's turnover events,
which include packing away the SysOp logs and other minor
maintenance processes) or the old record can be added to.
Data file IEMSI.DAT -- -- -- -- -- -- -- -- --
The IEMSI.DAT data file is created in the \temp directory only if
a user connects with IEMSI. The file contains the IEMSI record as
Telegard has processed the incoming IEMSI information. At the end
of this record will be the actual buffer dump of the IEMSI
handshaking package which was received by the system - just in
case for some reason it is required or needed.
The IEMSI buffer/protocol contains several fields which Telegard
can not process (either lack of functionality, or no real purpose
to process). See the record structure for specific entries which
Telegard will not do anything with.
Data file LANGUAGE.DAT -- -- -- -- -- -- -- --
The LANGUAGE.DAT file contains entries for all active languages
that have been defined for the system. There are a couple of
fields of note which should be looked at when searching to load
menus or to display text files -- Telegard searches the language
specific entries first, and then if that fails, looks at the
default entries (if the language is setup to do that):
LANGUAGEREC RECORD CONFIGREC RECORD
YES
START -> Is path field empty/null? -----> Search for file in
| | default path (MenuPath/
NO | | TextPath).
| | File found? -----> Use
Search for file in defined | | YES
YES path (MenuPath/TextPath). | NO |
Use <----- File found? | |
| | File does not exist.
| NO |
| |
Check defined boolean for |
further searching. |
(CheckMenus/CheckText). |
Boolean = TRUE --------------'
|
NO |
|
File does not exist.
The actual language text is stored in the *.TGL file with the
filename LanguageRec.Filename. There are more considerations for
Telegard 3.0 with it's new multilingual capabilities; see the
section titled LANGUAGE CONSIDERATIONS.
Data file LASTON.DAT -- -- -- -- -- -- -- -- --
The LASTON.DAT file is kept in sequential order - the last record
is therefore the newest record, and the first record is the
oldest.
The LCallers record is written when the user first logs on. The
LCallers.Caller field is used to update the record when the user
logs off to include the logoff time and download totals.
If writing a utility which handles this information, please
observe the HIDDEN flag for any records that are going to be
displayed, and only display to the users with Co-SysOp ACS or
above.
Data file LEVELS.DAT -- -- -- -- -- -- -- -- --
Telegard maintains the validation levels in a file called
LEVELS.DAT. Internally to Telegard, the levels are referred to as
A-Z (i.e. 26 levels can be stored in Telegard). Record #0 refers
to Level A, and record #25 refers to Level Z. All the level
records are created in the file at once, even if they are not
used.
If LevelsRec.Desc field is null, the level has not been defined.
Data file MACRO.DAT -- -- -- -- -- -- -- -- --
Telegard stores the macros in this file. When searching for a
users macro record, check UserRec.MacroPtr and check there for the
macro offset. If this record does not match the correct User-ID,
then set the UserRec.MacroPtr field to -1. The order of the
macros in the array is alphabetical; ^D,^E,^F and ^R.
Data file MODEM.DAT/MODEMnnn.DAT -- -- -- -- -- --
Modem information for single node boards is stored in MODEM.DAT.
For multinode boards, the default modem information is stored in
MODEM.DAT (will be used for all new node entries). Each node uses
the file MODEMnnn.DAT where nnn is the node number of that node.
Telegard 3.0 uses VERBOSE result codes instead of NUMERIC result
codes. This should make the configuration of modems considerably
easier. If Telegard detects the ModemRec.Code_ARQ string in the
connect string then the connection is considered Error Correcting.
The modems are very sensitive to the ModemRec.Delay* fields;
adjusting these fields even slightly will affect most modems
considerably.
Telegard 3.0 stores the IRQ/Base I/O address for the port. These
are purely informational for Telegard - since TG uses the fossil
drivers, non-standard ports are addressed through the fossil
driver. However, GSZ/DSZ and several other protocols access the
ports directly and may need this information on a per-node basis.
For more multinode considerations, see the section titled
MULTINODE CONSIDERATIONS.
Data file NODES.DAT -- -- -- -- -- -- -- -- --
Stores information on the node configuration (acs, logon times,
phone numbers) and the whos online information for each user.
The status text can be freely modified (it's displayed with an
MCI code in the language file) -- as long as the statusid is
correctly maintained for each node.
Data file SHORTMSG.DAT/INODEnnn.MSG -- -- -- -- -- --
When searching for short messages to a user, search the file for
matching user-id fields -- the messages are in no particular
order, so the entire file will need to be sequentially searched.
The status ID should always be 0 for all external Telegard
utilities. Telegard itself uses this field for internal transfer
stubs, and using a value other than 0 for external utilities can
potentially cause many problems.
To send a message to an online user for them to read, write the
message to the appropriate INODEnnn.MSG file. If the user logs
off before the message is received, it will be deleted.
To delete the short message (when deleting a user or after the
message has been sent to the user), set the ShortMsgRec.UserId
field to 0 (instead of the normal -1 as most other deleted records
do).
Do not delete short messages from the INODEnnn.MSG file. Telegard
will clean these up as neccessary (on the next login to that
node).
Data file TIMELOG.DAT -- -- -- -- -- -- -- -- --
The TIMELOG.DAT data file contains the time tracking graphs that
each of the nodes maintains. For single node systems, Record #0
is the only record that is maintained. For multinode systems,
Record #0 is for Node#1, Record #1 is for Node#2, etc.
The actual format of the data is not too significant, however:
TimeLogRec.BusyPerHour contains the number of minutes in
each hour breakdown that the node has been active for all
days during the week.
TimeLogRec.BusyPerDay breaks down that information into 20
minute chunks for separate days of the week. Both fields
should be updated, rolling time past midnight if necessary.
While it is possible to calculate the TimeLogRec.BusyPerHour
fields from the TimeLogRec.BusyPerDay fields on the fly, that
wasn't the way that I implemented the structure (grin).
The TIMELOG.DAT data file can be deleted to restart the time
tracking variables -- or alternatively the entire record can be
set to null (fill all fields with chr #0).
MULTINODE CONSIDERATIONS
Telegard 3.0 opens all of its files in SHARE mode -- this means
that for a multinode environment to exist, SHARE (or the OS
equivalent) must be loaded. Telegard 3.0 opens most of its files
in fmReadWrite+fmDenyNone mode, although occasionally Telegard
uses fmRead+fmDenyNone and fmWrite+fmDenyNone. When files are
being critically updated, Telegard will open the file in fmDenyAll
mode (and will only keep the file open a short period of time).
A few circumstances arrive where Telegard 3.0 actually LOCKS the
file contents; JAM, Squish and the SYSTEM.DAT file are all cases
where this is done. For JAM and Squish, follow their respective
development documents, and for SYSTEM.DAT see the information
earlier in the document.
When opening files in a multinode environment, there are a few
things to keep in mind:
(a) Open the file in fmReadWrite+fmDenyNone unless there
is a specific reason you need to open the file in
another file mode.
(b) All data files should be open as _little_ as possible
(i.e. do not keep a file open while waiting for input
for example).
Utility authors should not ignore the existence of multinode
systems -- if your utility can't co-exist in a multinode
environment, please specify it is for SINGLE node systems or for
systems which are shut down ENTIRELY for a maintenance event (i.e.
no nodes are left active during maintenance).
LANGUAGE SUPPORT
Telegard 3.0 introduced multilingual support for the BBS -- this
means that different language strings can be setup and chosen by
the online user to use.
While it is not necessary for Utility authors (esp. those with
with interactive doors) to support multiple languages, keep in
mind that your product could be used on several different language
platforms, and multilingual support could be beneficial.
There are two common ways to handle multiple languages; the first
is to create a text file which is loaded on the fly (good for
small amounts of text), and the second is a data file which stores
different strings for different languages. Both methods are
equally useful, and both have their drawbacks. If you intend on
supporting multiple languages, the choice of which method to use
is entirely up to you.
You can load the current users language from their user record,
or you can have it passed on the command line with MCI codes. The
choice there is also up to the utility developer.
Telegard language file structure is *not* distributed at the
present time. If you need text strings from a Telegard language
file, please get in touch with the development team for more
information. The text strings are not always at the same file
offset, so seeking into the file to get around the lack of
structures will not consistently work properly.
TEXT FILE DEFINITIONS
Telegard 3.0 added the ability to import text file definitions for
archivers, protocols and modem setups. This allows standard
configurations to be freely distributed and installed without the
hassle of playing with the setup, or having to input all of the
field information. The text file definitions can be imported and
exported by Telegard 3.0, and the format is freely available so
that distribution can easily occur.
SysOps are encouraged to release new archive and protocol setups
(provided they work, naturally) so that other SysOps can benefit
from this new distribution method.
The three formats (ARCHIVE, PROTOCOL and MODEM) are all discussed
in their own document files (TEXT_ADF, TEXT_PDF, TEXT_MDF
respectively).
UTILITY DISTRIBUTION III
_____________________________________________________________________
DISTRIBUTION TYPES
When deciding to distribute your archive, there are several
considerations you must make -- mainly dealing with what you want
in return for your utility, etc.
There are several distinct types of distribution, and several that
fall in between categories - FREEWARE, SHAREWARE and COMMERCIAL
SOFTWARE. There are too many misconceptions and other conditions
to describe each type of distribution in this document; the best
method is to determine what you want, find a similar product and
look at how they marketed their product.
THE DISTRIBUTION ARCHIVE
Most distribution archives are packed in such a way that SysOps
can quickly look at what is inside and determine that they are
getting everything that they need. Some SysOps will not even
look at your program if they are in any way suspicious of the
contents of your distribution archive. The following hints are
aimed at helping you as a utility developer get the most out of
your efforts.
THE ARCHIVE FILENAME
Almost all utilities will require more then one release version,
so version numbers come in handy so that SysOps and users know
what version of the program the current archive contains. Thus,
the version number normally finds its way into the filename of the
archive, either as FFFFFFnn.EEE or FFFFFnnn.EEE depending on how
the version numbers are represented (the first is for Vn.n, the
second for Vn.nn -- your choice, personal freedom). The filename
could also contain many variations on the above format, such as
FFnnnFFF.EEE, etc. If at all possible, shorten your archive
filename and add the version number to the name -- that is the
best way to ensure that SysOps will take a look at your file.
And if you generally think your utility will not have any more
versions, give it another thought -- most do.
THE ARCHIVE FORMAT
The most popular archive formats are still ZIP and ARJ. Other
formats, such as RAR and LZH have followings in certain regions,
but may not necessarily be used everywhere your archive may
travel. You are best to stick with the most popular archive
formats -- if you really want to deviate, consider self-extracting
forms of your archive choice.
Also, if you are considering a self-extracting form, it is
suggested that you first pack your archive with the
self-extracting header, and then pack that .EXE file (and any
release notes) in a .ZIP file. Your release notes should be
separated from the your self-extracting .EXE file -- some SysOps
read the release notes to determine if they want to look at the
file further. If they only see an .EXE file, they might look the
other way.
THE ARCHIVE CONTENTS
All pieces of your program should naturally be contained in your
archive (this includes all necessary data files, text files and
of course the executable files).
Your archive should also contain some form of documentation. Even
if your program doesn't do much, you should include documentation
which at least covers the following topics: (a) a brief
description of the program, (b) what files and versions of those
files are required -- if any, and (c) contact information for
reaching the author. The program documentation normally takes on
the name of the executable file with the extension of .DOC,
although this isn't a standard by any means. Some software
distribution networks and BBSes refuse to carry files without
documentation -- consider this, and even if it's only a small 1k
file that you write, at least it ensures that your archive won't
be immediately deleted.
Last minute release information should be put into one of several
files; README.DOC, UPDATE.DOC or RELEASE.DOC. These files are
commonly searched and read before any other miscellaneous files
are touched, so putting last minute information into these files
is a good idea.
If you wish to ensure that no files are added or removed from the
archive, then a text listing (in both your main documentation and
a separate file called PACKING.LST) of all the files in your
archive is a good idea.
If you wish to ensure that your archive is always described the
way that you want it, a FILE_ID.DIZ file is a great idea (in fact,
many popular BBSes now import the file as the file description,
regardless of what the uploading user types in). The FILE_ID.DIZ
file is a text file which describes your program. This file has a
specific format; the maximum number of characters on any line is
45, and the maximum number of lines is 10. PLEASE DO NOT DEVIATE
FROM THIS STANDARD - I REALIZE THAT OTHER AUTHORS HAVE AND JUST
BECAUSE THEY DO DOES NOT MAKE IT RIGHT. IF YOU WANT TO ENCLOSE A
FILE_ID.DIZ FILE, ENCLOSE IT IN THE PROPER FORMAT.
Some authors like to time stamp all the files in their archive so
that all the files have the same date/time. This makes it very
easy for the author to check and see what files have been
changed/added to the archive, by checking the file date when
unpacking. This obviously isn't something that everyone needs to
do -- personal preference again.
Beyond that, archives are archives. Many authors have their own
personal ways of building their archives and of setting up their
distribution files. You can be just as free with your decisions,
just remember that when it comes to distributing your archive, you
may be limiting your distribution if you don't put together a
complete package.
DISTRIBUTION
There are several ways of distributing your final archive. These
ways will be discussed below so that you can determine the best
way of getting your file out into the field.
ADVERTISING
The best place to advertise new utilities for Telegard is in a
Telegard support conference. The participants of the conference
are mostly Telegard SysOps or other interested parties, and thus
advertising in the support conferences will ensure that you hit
your best (and most interested) audience. TG_SUPPORT (our
international Fidonet conference) is probably the best place to go
for this advertising. Please note however that you should
advertise your program only once or twice; advertising it daily
will unnerve many people and may turn them off of your product.
FILE REQUESTS/MAIN BBS DOWNLOAD
Setting up your archive for File Request (mailer) or BBS download
ensures that people will be able to get to your file. The problem
with relying on this method alone means that your utility spreads
only through direct contact (i.e. the interested SysOps must call
your system to get the file). This results in a very systematic
and slow diffusion of your program -- although it's still
recommended that you do this, there are other methods you should
also consider.
SOFTWARE DISTRIBUTION NETWORKS
Software distribution networks distribute your file for you; you
send it in to a site, which sends it to connected systems, and
those systems send it to connected systems, etc. This ensures
that everyone who is connected to a specific SDN gets your file
automatically (and in many cases, the file is automatically added
to their file bases for download as well). This method of
distribution means that you program will be spread in a very
hierarchical way, but will spread quickly and automatically -- two
things new releases generally benefit from.
There are several different distribution networks out their to
distribute your utility. Most networks have different categories
for different types of files; so when choosing a network, make
sure that they have an area for your type of utility (i.e. a
Telegard area, or a BBS utility area).
The Telegard Development Team has created a software distribution
network on the fidonet filebone which provides national and
international distribution through Fidonet channels. This area
has the tag name TG_SUP and should be available from your Fidonet
hub. If you would like your file distributed via this SDN, please
see the section on DISTRIBUTION BY THE TELEGARD DEVELOPMENT TEAM.
FTP (FILE-TRANSFER-PROTOCOL) SITES
The internet provides a service called FTP, which allows for the
collection of files over the internet. This ensures that people
can access your file without cost (except the cost of their
Internet provider, if any) and at any time of the day, without
worrying about busy signals, etc. Although I'm not aware of most
of the anonymous FTP sites, if you upload your utility to one of
them into a /Telegard/ area, I am sure that you will get some
internet users grabbing your file.
The Telegard Development Team presently operates one official
FTP site, available through FTP at ftp://filebank.com. If you
would like your file distributed via this site, please see the
section on DISTRIBUTION BY THE TELEGARD DEVELOPMENT TEAM.
DISTRIBUTION BY TELEGARD DEVELOPMENT TEAM
The Telegard Development Team operates many distribution sites
which can help distribution your utilities (TG_SUP, ftp site and
the development sites). To have your file distributed via our
distribution methods, please refer to the following "to-do" list:
(a) Ensure your program works (test!)
(b) Include complete documentation and contact information
(c) Include FILE_ID.DIZ file
(d) The preferred archive format is ZIP -- if you pack
in another format, please consider double packing so
that you file is ultimately in ZIP format.
(e) Upload to a Telegard Development Site, let the SysOp
know the file is there, and provide a simple one-line
description (this is for the actual SDN distribution)
(f) The file will be hatched into the TG_SUP as soon as
humanly possible. Within 36 hours of that hatching,
the file will be placed on the FTP server.
OTHER QUESTIONS IV
_____________________________________________________________________
If you have any further questions on topics discussed in this
document, or even topics which were not discussed, please feel
free to contact the author of this document or get in touch with
one of the Telegard Development Sites.