The Obide plugin is principally divided into four areas: 1. source data retrieval; This is the set of functions to supply data for the source code scanner and parser. 2. source code scanner/parser; This is the subsystem that parses the source code and builds an internal data representation structure ready for fast access. 3. data requisition system; This subsystem requests data from the internal representation using the query mechanisms provided by the internal database. 4. user interface. This subsystem issues requests on behalf of the user and displays the query results in a helpful form. 1. Source data retrieval The source data retrieval subsystem gets the required module name and produces an instance of Project.Reader object which reads that module's text in the Read2 method implementation. The readers are currently implemented in the Tools module: ScintillaReader reads data from a Scintilla handle, and StreamReader reads from Streams.Stream (i.e. disk or memory files). To help decide whether a module requires re-parsing there is the Project.Locator object. It stores the system-dependent information on the module location and can create a Reader instance for the module. Some modules may not have a Locator assigned, e.g. if a module is parsed from a memory stream or from a Scintilla handle. Usually projects contain a number of modules, and when a module is parsed the imported modules are often need to be parsed as well (if they aren't present in the parsed database or if the assigned Locator reports that they need to be updated). To find a module by its name a special procedure is used of the type Project.GetModuleLocatorProc (it may later be replaced with an object type). The procedure attempts to find the module and returns the module's Locator on success. The Locator instance is subsequently used to create Reader instances. The implementation can be seen in Tools.GetModuleLocatorProc. 2. Source code scanner/parser The scanner/parser fits entirely in the Parse* procedures of the Projects module. It produces a tree of objects, all inherited from the basic Projects.Item. The database of all the currently parsed modules is accessed from the global 'root' variable (not exported). Whenever a new module is parsed, it replaces the old module of the same name in the database. For every variable (Var) object its type is stored as a reference to a Type object instance, which can be one of the following extensions: TypeItem, TypeRef, ProcType, ArrayType, PtrType, StdType, Struct. I'd like to specially note the TypeRef object, which represents a weak link between parsed modules. All other objects in the parsed database only reference Items located within one module, but a type reference may be referring to a type defined in another module, e.g. "r: Tools.StreamReader". In this example .modName = "Tools", .refName = "StreamReader". This weak linking via module name (as opposed to hardlinking via a Module object pointer for the parsed module) allows for three features: 1) the module in question may not be present in the database, so there is no pointer of type Module to assign to the link, but when it's needed, it may be added later by passing the .modName to the GetModuleLocatorProc, thus creating the Module instance; 2) the weak link, therefore, always points to the most recent Module object that exists in the parsed database; 3) it does not prevent old instances of the Module object from being collected by the Garbage Collector (GC) after it has been kicked out of the global Projects.root chain. Each Parse* procedure, except for Parse, expect the current value of the 'sym' variable to be the next symbol to parse, and after execution they leave 'sym' to be the first unhandled symbol, ready to be consumed by another Parse* procedure. If a procedure can't handle the current symbol, it won't advance the current state of parsing. Parse* procedures retrieve the next symbol by calling the GetSym procedure, or, sometimes, CheckSym. When implementing parsers, special attention must be given to the looped symbol reading: all loops must terminate not only when a suitable symbol is found, but also in case of OPS.eof symbol. Otherwise the parser risks locking into an infinite loop. 3. Data requisition system The parsed objects database is contained within the Projects module, starting from the global 'root' variable (not published). Currently there is no notion of projects, so all modules are considered to belong to the same project, and are only distinguished by name. This means that if you are working on two projects simultanously and have a module with the same name in both, those modules will be considered the same by the database. All item types are published, and all object fields are exported as read-only to prevent inadvertent damage to the database. The basic interface requires a client to use the Project.GetModule procedure, and then allows the client to browse the entire module's contents. The GetModule procedure returns either NIL (if the requested module is nowhere to be found), or the most up-to-date version. If the version found in the database is out-of-date, the module is reparsed using the supplied locator and the database is updated before GetModule returns. The GetModule procedure and the read-only field access are the only two features required to fully support any kind of project data mining, provided that Locators give access to the underlying source data. The Parse procedure is also published, even though it does not access or update the database. It is not strictly necessary, but it's handy for the use of Obide as a text editor plugin. Namely, it allows parsing of the modules which have no Locator, e.g. the ones only existing in the text editor buffer, or possibly loaded from disk, then edited, but not yet saved. Such a module does have a file associated, but cannot be accessed in its current state using the file Locator. The contents of such a file are always accessed via the Parse procedure, and all the modules it refers to are taken from the database, which reflects the current on-disk state of files. The Parse'd module may have a new procedure or variable declared, but until it's saved to disk the other modules will be unaware of that, so the new item won't appear in the module's context list. There are a number of utility proecedures to aid the data retrieval beyond the basic interface. Projects.NamesLength and Projects.NamesToString help make a list of .name field contents of a Projects.List object. These procedures should be moved away from the Projects module, they are only included to avoid inefficient Length implementation: the efficient implementation would require a VAR-parameter, but Item.name is exported read-only and can't be passed by reference. Projects.TypeToDeclString returns a string representing the declaration of a given Type object in a user-friendly form. Projects.ListSubitems returns a list of subitems for an Item, i.e. fields and methods for an identifier, or a list of global items declared in a module, etc. This procedure generates the data for autocompletion lists. 4. User interface The user interface subsystem relies heavily on the use context. Currently Obide is implemented as a plugin DLL for Notepad++ editor (both ANSI and Unicode builds are supported). The UI modules are Obide, ObideU, Npp and Main. The first three are just bindings used to register the module in Notepad++ and invoke the Main module's procedures via the appropriate shortcuts. The Main module is the core of the plugin. It takes the context of the Scintilla text editor (the current document, caret position, etc.) and translates it into the Project database request. The request is performed and interpreted in the Main.IdentifyContext procedure, and then presented to the user in the form of autocompletion lists, code hints, and other means provided by the Scitilla component's UI functions. Some other useful features, such as code navigation, are also implemented in the Main module. There is an ongoing attempt to use database mining capabilities to provide features not directly supported by Scintilla's UI functions, e.g. to display the type extension graph or module import graph. These attempts usually generate some text representation, which is then output to a new document. It is clear that such features would also be suitable for a command-line tool, which is just another form of UI.