3Depict
Valued point cloud visualisation and analysis
________________________
User manual
Website:
http://threedepict.sourceforge.net/
Version:
0.0.22, Dec 2018
3Depict
Valued point cloud visualisation and analysis
________________________
User manual
Website:
http://threedepict.sourceforge.net/
Version:
0.0.22, Dec 2018
3Depict is an open source computer program designed for the analysis of point clouds with an associated scalar value. The software is designed around interactive data analysis, with a view to combine rapid feedback, ease of use and flexibility in a single system. At time of writing, 3Depict is in the so-called “alpha” prototyping stage, and should be used where helpful, but may contain rough-edges.
3Depict is designed purely for post-processing of 3D point data, and was originally primarily targeted to users of Atom Probe Tomography. Other users (e.g. in astronomical, geospatial or digital preservation fields) may find the program useful, and are encouraged to seek assistance.
3Depict attempts to fill a perceived need for freely available flexible point data visualisation. This program is designed to manipulate and modify point data in a way which the author has otherwise not found a suitable program to do.
With this program, point data can be visualised using a fully implemented camera system, edited with directly interactive objects, and subjected to various analysis algorithms. A real-time plotting system is also provided to generate analyses of your data on the fly. External programs can be engaged as part of the system to create new analyses that “clip into” the analysis.
Open source programs are programs which distribute not only the executable code, which is understood by the computer (so called machine code), but also provides the version of the program as it was written by the developers as well. This provides external users with the possibility of modification or verification of the program behaviour, either by themselves, or by engaging a third party. With the source code one can verify the correctness of the system, alter behaviour or otherwise modify the program, or even reuse sub-sections of the program elsewhere.
Modifications to the program itself may include migrating the program to newer or older systems, adding new functionality, or correcting errors in the program implementation.
To provide the user with these capabilities, the program is distributed with a so-called libre copyright licence. The program is distributed at no cost to the end user, and the copyright attached to the program explicitly allows modification and re-distribution (copying) of the program to other parties.
Note that there are restrictions on what may be done with the program, for example it is in violation of the licence to claim ownership of the program, or to use technical measures to prevent access to the program, or modification thereof. The licence used in the program is a generic one shared by many free (as in freedom) software programs.
If you have been charged for this program, it is suggested that you request a refund and obtain a free copy from the main website, as listed on the front cover of this document. If you wish to have the full licence details (GNU General Public Licence Version 3 (or any later version)), please see the COPYING file distributed with this program. If this is not available, please see the project website, or perform an Internet search for the licence name.
Due to the design of the program, the program should run under Linux, Mac, BSD and Windows machines. The program does not rely on CPU specific features, and thus should be able to be run under x86, x86-64, arm, or whatever. Basically, it should run just about anywhere. Every effort is expended by the author to ensure that the program can be run on as many devices as possible; if your platform is not supported, it may be possible for either you, or the author to generate executables for your system. See the section “Getting Help” for contact details.
The minimum requirements for running 3Depict are not known. The author wrote a substantial portion of the program on a machine with only 4 and 12 GB drives, and a 1.6 GHz processor, which normally runs at 800 MHz and has 1 GB of RAM. There is no clear reason that it would not run on even lower-spec machines. Whilst a higher spec machine may run the program faster, intelligent use of the programs “filter” system may allow for complex analyses even on low-end machines.
If you are experiencing 3D graphics problems, first ensure that other 3D programs do not experience the same problems. Otherwise, please contact the authors for assistance – there should be no requirement for vendor-specific hardware. Note however that the exact appearance of the 3D view is dependent upon your hardware, and may have small changes between different platforms.
Note that whilst every effort is made to ensure that the program will run on a variety of systems, small system-specific quirks may be evident, particularly on platforms to which the authors do not use regularly (e.g. windows). Secondly, due to slight differences between platforms some functions may be remapped to other mouse/key combinations.
Mac:
Windows:
Assistance with this program may be freely obtained over the Internet at http://threedepict.sourceforge.net. Questions regarding use of the program, feature or bug reports will be attended to as soon as possible. Contact options include email (via the online web-form), or an online forum.
If the program crashes in a predictable manner (i.e. you know how to trigger it), this is a bug and needs to be fixed. Please report the bug in this case, so we can fix it as quickly as possible. If the program crashes in an unpredictable fashion, please still report it as best you can, and we will try to fix it if we can isolate the problem from the description. For advanced users, we would appreciate backtraces, packages, and any other relevant information in both of these cases.
This program was written by D. Haley, in his spare time. A. Ceguerra provided additional development from Version 0.0.2 and provided assistance with debugging and fixing the Macintosh version, and providing executable versions of the program for OSX in 0.0.1. A London provided debugging assistance, and from 0.0.21 has developed features in the filter system.
For the more visually inclined, screencasts of the program have been created, and are available on the project website. These videos exhibit basic use of the program for various simple analyses. At time of writing, the only literature available for the program is this document, and the online screencasts. If you have questions, please contact us through the website, where we will reply as soon as possible.
3Depict takes time to develop, and no doubt could be better than it is now. However, this doesn’t all just magically happen – people have to put the work in. Development time by the authors is split between testing the program, reproducing bugs, coming up with new ideas for program changes, editing documentation, making pretty pictures, maintaining websites, and even developing the program.
We would always appreciate assistance with this work. You don’t have to be able to write computer programs. For example, we would like to translate the program into other languages. If you can translate a spreadsheet table into another language, this is helpful. If you can work out what triggers particular bugs, this is helpful. If you can improve this document, this is also really helpful. Of course, if you can program (C/C++) and are willing to help, grab a copy of source from our website contact the authors, because a little code goes a long way.
This program is distributed under the GNU General Public Licence Version 3 (GPLV3+), an open-source licence. Information on the copyright of this program is available under the COPYING file in the program directory, or online (e.g. http://www.gnu.org/licenses/gpl-_3.0.txt or https://en.wikipedia.org/wiki/GPLv3).
The basic premise is that you may copy the program, modify and distribute such modified versions or derivative works only under the same licence, whether a part or the entirety of the program is used. The licence forbids technical restrictions on users further redistributing the program.
The installation method for the program depends upon your chosen operating system. The most up-to-date notes are available on the project website. It is highly recommended that, in general, you do not simply download random programs from the Internet and execute them if a version is available in a trusted software repository. At time of writing (Oct, 2013), installers are available for windows, Debian and Fedora-like linuxes, and some versions of Mac OSX.
The program interface consists of three different views. On the left, there is the data, cameras and tools panes, with are used to generate data for visualisation, and to provide an interface into changing properties in a structured manner. On the right, the view is split into two sections; at the top, there is the 3D view. At the bottom are the plotting, raw data and console output panels.
Each pane may be hidden, either by double clicking the “sash” between the two panes, by selecting the respective item from the view menu or by its keyboard shortcut key as listed in the menu.
At the very bottom of the program, a status bar is shown – here messages are shown to provide hints on how to use the program, or to communicate information relating to the program’s internal state.
Understanding the filter tree is very important to being able to use 3Depict to meet your needs. The reason it is called a tree, is because it is a graphical representation of the mathematical “Tree” - where each node in the tree can have a “parent”, or several “child” nodes. In 3Depict, each node in the tree is called a “filter” — the filter modifies data coming from its parent in some manner specific to the type of filter, and the options chosen for that filter.
The filter tree is quite important in 3Depict, and the ability to change it is also a very powerful tool. You can copy, move (by dragging the nodes) or even store sections of the tree (by using the “stash” panel) , allowing you to assemble the tree in whatever manner you find useful to your analysis. Using the tree is discussed in more detail in Section 4.2. You can also change the names of the filters to help you identify which filter is which, simply by clicking once on the label.
The filter tree may, depending on how it is laid out, show small warning symbols next to the tree - these usually indicate that you have built the tree in a manner which may be either non-helpful (e.g. the output of one filter cannot be used by the other), or that the configuration might generate misleading results. There are currently two levels for this - “Error” and “Warning”. Warnings can be ignored safely if you know what they mean - for example you might be attempting to calculate the density of your point cloud, but you have only partially loaded the data – 3Depict will detect this case and issue the warning.
Not all incorrect configurations are warned about – you must think about the construction of the tree carefully when working with the program. Different structures can generate radically different results.
Each filter in the tree has a list of properties that are generated by the filter. These are listed below the filter tree. The properties of the currently selected filter are displayed, and can be edited. Only the relevant properties for the current filter mode will be displayed. Enabling one property, or changing the value of a given property may cause new properties to appear dynamically. For example, enabling a cutoff option in a search algorithm will cause the properties for the lower and upper bound to appear.
Editing any property can trigger an update - also known as a “refresh”. If the “automatic refresh” option is enabled, the program’s state will be recomputed. If a property is edited whilst “automatic refresh” is disabled, then the property will usually have no effect on the final state, and a manual refresh will need to be performed once property editing is complete. Disabling automatic refresh is most useful when editing several properties, and you don’t want the computer to automatically recalculate an intermediate result.
There are several types of properties, including colours, 3D points and file paths. Each of these properties can have a specific input method. For example, colours can be inputted using the colour selection dialog, or alternately, these can be typed in by using some common colour names, such as “green”. Point properties also have the ability to enter them in different ways. By default, 3D points can be inputted using comma separated values, such as ”1,2,3”, to specify the Cartesian coordinates for the property. However, it is also possible to use a different notation to enter these points in as spherical coordinates. Specifically, if one inputs <1,2,3>, this no longer corresponds to x=1,y=2,z=3, but now corresponds to an ⟨r,θ,ϕ⟩ spherical coordinates.
It is possible that setting properties in one filter may conflict with another setting elsewhere in the tree. In this case, it may be detected by the program and a warning may be shown to signal this case. As an example, if one filter has a sampling property set, than a calculation that insists on total counts may cause the final result to not be indicative of the true value of the original dataset, and a warning to this effect may be shown.
The 3D view is used to show the three-dimensional objects generated during a data analysis, and provides a direct method of interaction with the 3D Scene. Through the use of the mouse (or other pointing device), the 3D view can be manipulated to change the view position and orientations. Some objects in the 3D view are interactive, and will be indicated by an overlay in the top right of the window when the pointer is on top of such an object.
The 3D view represents your camera into a 3D scene of your construction; it is by manipulation of cameras that the view is interacted with; so you may zoom, orbit, pan, roll or swivel the camera view. If you are lost at any time, you may reset the view by tapping the space bar. To change the axis along which the view is reset, hold the Ctrl or Shift buttons whilst resetting. Double tapping the space bar will cause the axis to be viewed from the reverse direction.
The basic 3D view consists of a “target” based camera, so when you move the camera, the camera will orbit around this target. To interact with a scene, hold down the left mouse button and move the mouse to control the camera.
The basic keys for controlling the camera move mode (left click) are1 :
For any motion, the Shift key may be used to increase the camera move speed. Scrolling on the window zooms in or out. For a perspective camera, zooming is performed by moving the camera closer to the object. For an orthographic camera, zooming simply scales the view, whilst holding the camera position constant.
The available plots are listed on the right hand side of the plot view panel. You can select the active plot from the list. The items in the list take their name from the filter from which they originates name (there are exceptions to this rule, i.e. composition profiles). Several plots may be drawn at once by holding down the Ctrl key when selecting the plot to draw from the plot list box.
Raw data is visible in the “raw” tab (Figure 5), and will show the output data from the selected plots, with the axis labels for each plot. The data can be saved to a file from this view.
Each filter may optionally generate console output. In the case, a text area will contain messages from the filter to the user. An example of the messaging area, and the messages are displayed in Figure 6. As can be seen in this figure, if a message has been generated from a filter, but is not the messaging area is not active, the console tab will display a small marker to denote new messages pending for review. The exact marker that is shown is dependant upon the operating system.
The tools panel offers several options on changes to the way the program operates internally.
Initially the program window will appear with only the default world axes visible. To provide a more interesting view, it is necessary to inject data into the program. To do so, select the File menu, and then select using “Open”. At time of writing, only two formats are currently supported. Firstly are “POS” files, and secondly are text files, each which consist of X,Y,Z and a values (usually mass-to-charge)2 . To load a file, navigate to an existing POS file on your disk. If you do not have a POS or text file, small example files are available on the project website, on the documentation page.
Upon selecting the file and then Open/OK, the file will be loaded into the viewport. Note that the entire file is not loaded, but rather a random selection of elements in the file.
Loading this file populates a small treeview on the data pane (at the left). This tree is referred to as the “analysis” tree, and each item in the tree is called a “filter”. The tree is responsible for producing the output data in the scene, and a good understanding of the behaviour of this tree is required to extract the maximum benefit from the program. Each item in the tree has a list of properties that can be modified. For example, the amount of data loaded by the “pos load” filter can be altered by selecting the “pos load” item from the tree, then in the grid below, entering in the new amount of data to load (you can set this to 0 to load the entire file).
Thus, each filter can be individually altered to change its behaviour. However, each filter acts upon the output of the filter that is a “parent” to it (in the case of not having a previous filter, each filter will act as if it had no incoming data). Thus the arrangement of each filter in the tree is critical to the output of the program. In order to modify the layout of tree, you may add new and move, copy or remove existing components of the tree. Changes to the tree, or any filter contained therein, may be undone using the “Undo” menu item, or with the keyboard shortcut Ctrl-Z. Each filter’s behaviour is outlined in Section 5.2. More information on the tree behaviour is given in Section 4.2.
Note that with every modification of the tree, the 3D scene and any plots will be recomputed. The time of computation is dependent upon the amount of data that is to be analysed, and can be reduced through sampling or volume restriction methods. By default, each filter may cache its own output, in order to speed repeated computations.
To delete an item, simply select the item to delete with the mouse, and then use either the Delete or Backspace keys on your keyboard. Note that clicking on an already selected item will activate the name edit mode. To exit this mode, press Escape.
New items can be added to the tree by selecting the filter to add from the dropdown box immediately above the tree. When selecting a new filter to add; an element in the tree must be selected, where the new filter will be placed. If there is no item selected, an error will be shown in the status bar.
Once an item is added, the filter tree is thus modified and a recomputation of the scene will occur. Approximate progress on the filter update is visible in the status bar. During an update, only limited interaction with the program is permitted. An update may be cancelled at any time with the escape key.
Several quick notes are provided here as examples of how to perform specific measurements/calculations. Whilst this is not an exhaustive list of measurements that can be made in 3Depict, this section is targeted towards new users who wish to use the program to perform quick or common measurements.
To load data, one must first have data to load in the form of either a ”POS” formatted file (see Section 8.2.4), or as a text file (using English decimal notation (.), four columns - see Section 8.2.5). To load the data, use the Open command in the File menu. Alternatively, one can drag and drop the file onto the program.
You may have an existing analysis file, which you can use to load both the data, and any associated analysis information (plots, clustering, clipping, etc.), which for example may have been undertaken by a separate user.
To load it, you require a “package” from the previous user, which will be a folder containing a XML file, and any data files that are required. As for loading data, you can either directly open the analysis with File→Open, or by dropping it onto the program.
Note that by default, the program will not load all the data in the file - a sampling will be performed. Careful use of data sampling will allow for a much more rapid and interactive analysis of large datasets - many of the algorithms running times do not scale directly with the size of the dataset. Reducing the number of ions can, in some cases, result in a significant reduction in run time (e.g. halving the number of ions for some algorithms can result in a run time of a quarter required for the full dataset).
The program can be used to mark particular sections of the spectrum as belonging to a particular “range” of values. Each value can be tagged with a specific name for the range, and an associated colour, which will be used to mark the points in the 3D display.
To perform ranging, you must first have a valid rangefile. As of time of writing, (July, 2013 - 0.0.14), the program is unable to generate these from the UI. It is possible to write the file by hand, or using a separate program - details on manually writing the file can be found in Section 8.2.3.
To perform ranging, the data must be first loaded into the program. The range information can be loaded in two ways, by dropping a valid rangefile onto the program, or by using the filter dropdown (Figure 7), whereby a window will open that will allow for the selection of a valid range file.
Once loaded, you can select the ranging filter and enable/disable ions and ranges you do not wish to see. By default unranged ions are not emitted from a range filter, so will not be seen unless “Drop Unranged” is unselected.
To see the mass spectrum for a selected data, the “spectrum” filter must be used. First load the required data (as per Section 3.1), then select the data filter in the tree, and select “Spectrum” from the filter drop-down. This will display the spectrum without any overlaid ranges. To get the desired signal/noise level, you may wish to either alter the sampling level in the Pos Data (Load Limit value), or disable sampling. Changing the spectrum bin width until the spectrum appears as desired is also recommended.
To display the ranged spectrum, simply use the sequence Data→Ranging→Spectrum, as shown in Figure 8. To see the ions outside existing ranges, untick “drop unranged” from the range filter. Note that as the data is converted into a spectrum, the 3D view will disappear. To see both the spectrum and the point data at the same time, use the configuration shown in Figure 9.
To display a composition profile, you first require a ranged dataset (see Section 3.3). Once done, first select the range filter, then choose a concentration profile from the drop down. Note that as the data has now been converted into a concentration profile, the point cloud will disappear (although the concentration profile cylinder is visible, and is computing the correct result). To see both the data and the concentration profile at the same time, use the configuration shown in Figure 10.
To compute the absolute counts of the number of ions that are visible in the dataset, use the “Ion information” filter, as shown in Figure 11. To compute the ion count, check the “count” box. To compute the dataset volume, select the “volume” checkbox, and the desired algorithm (For algorithm details see Section 5.2.4). The results are displayed in the console window (Figure 11).
To generate iso-concentration surfaces, or to create 2D slices in your data for visualising information such as concentration fields, a voxelisation must first be conducted.
Load some ranged data (Section 3.3), and then select the range filter and choose “Voxelisation”. This will cause the dataset to disappear until you configure the voxelisation parameters appropriately. To compute a concentration field, change the normalisation mode to “All ions (Conc)”, then select the ions that are to be included. To visualise the result, change the “representation” mode to either isosurface or 2D slice. The upper and lower bounds of the 2D slice are auto computed.
However, for the isosurface one must choose the value that the isosurface is spanning. For non-concentration modes, after computation of the voxel field, the upper and lower bounds of the field are shown in the console window, and can aid in selecting the desired isosurface value. For normalised modes (i.e. concentration), one would set the values between 0 and 1.
Filters form the key component of the program. These are the tools by which data is analysed and modified, in order to generate the visual representation that is needed by the end users. The basic idea behind a filter is that each filter may perform arbitrary operations on “data streams”. These data streams are sent to and from each filter, flowing through the tree.
The basic idea of a filter is illustrated in Figure 12. 3Depict’s flexibility is that these filters can be arranged in any way that makes sense to the end user. There is no restriction on placement of filters – some placements may be totally useless, others may be exceedingly useful. It is up to the creativity of the end user to determine whether any single arrangements meets their needs.
The tree is a flexible and powerful system for constructing your own analyses, after some use this will become a familiar and readily modifiable system for performing your analyses, however the initial structure of the program may take some getting used to. If you are familiar with programs such as Paraview, you may already be familiar with this concept.
The filter tree essentially is a system for injection, manipulation and display of the data in the program. The tree becomes an “assembly line” for the view of data in the 3D and plot views. The nodes of the tree are the filters that act on or insert data into the analysis. Each node in the tree may be considered in what is called a “parent-child” relationship. Each element in the tree (except the first) has a “parent”, and thus may have their own “child” elements. Each “parent” may, in fact, have many children. Data may be considered to propagate from the “root” of the tree downwards, with each filter in a direct line somehow modifying the data from above in some way. When data reaches the end of the filter tree it is “picked up” by any of the 3D view, plot or console panels, depending upon the nature of the data.
The basic method for data flow is that a parent gives a copy of the data it has processed to its “children” to modify in some way. Each “child” has its own copy3 of the data from the parent, which it modifies. In turn this child then gives a copy of the data to each of its own children. If a filter has no children it then passes the data to either the 3D view, the plot view or the console view, depending upon the data type.
Using this method, one may create a variety of different analyses; for example, one may wish to subsample data before performing a time-consuming spatial analysis, or one may wish to clip the data to remove unwanted sections before generation of a value spectrum. The flexibility of the filter system supports this concept.
Note that items in the filter tree can be moved. You may move any filter to a new parent by dragging with the mouse. In order to copy instead of move, hold down the Ctrl whilst moving to duplicate the filter, rather than moving it. You can “splice” the node to a new location (change its parent, without moving its children) by holding down Shift whilst dragging.
You may also rename filters in the tree by clicking (long-click, or three-times slow). The filter name may be used by the filter to generate its output, e.g. spectrum plots will take the plot title from the filter name.
Instead of enabling or disabling sections of the tree, the program supports “stashes” as a place to put sections of the analysis tree for later use without using them in the analysis section. To create a “stash”, select a section of the filter tree to “stash”, then in the “stashed filters” dropdown on the data tab, type the name of the stash you wish to create (this is up to you), and press Enter. Once done, a duplicate of the subtree specified (i.e. all the filters below the selected one, and the selected one too), is made. This process is shown in Figure 14. You can view the contents of the stash by selecting the button next to the stash dropdown, and you may delete stashes however you cannot edit them.
To use a stash, select a filter in the tree and then click the dropdown button on the stash combo box, and then select the stash you wish to use. This will place the stash as a child of the selected filter. Note that the stash can be used multiple times.
Any plots generated by the filter system are displayed in the plot pane. It is possible to zoom or pan the view as required by dragging or shift dragging the plot respectively. Double-clicking the plot returns the plot back to its original scaling.
The associated numbers used to generate the selected plots are shown in the “Raw” tab. Note that plots can contain “regions”, such as generated by a range file. In this case, each region may be manipulated in-situ, by dragging the regions sides, or its centre to alter or move the region respectively. These modifications will be propagated back to the original filter.
Each plot is either logarithmic, or linear in scaling. Mixing these two types of plot will result in the y-axis stating that there are mixed data types in the plot. The log/linear mode is determined by the filter that generates the plot. Note that due to internal limitations (fixed plot palette in the underlying library), the colours observed in the plot may be slightly different from those specified by the filter.
To fully understand the camera model, it is necessary to understand the parameters in the camera property tab. Initially there is only the default camera, which is unnamed. By entering in a name for the camera, you can access the properties for that particular camera. By entering in more names, you can create multiple cameras, saving the position of existing cameras as you go. This can allow you to jump between different camera views as desired.
One can select the position of the camera, a position that the camera is always looking at (target), the camera “up” direction, and the field of view. Furthermore, the camera type (perspective or orthogonal) can also be selected.
With the exception of the field of view, these parameters are dynamically modified when interacting with the 3D scene (see section X). The camera field of view, however requires special mention. The field of view of the camera is the angle that the camera look at. Human vision is around 120*, and is much narrower for suffers of tunnel vision (say, 30*). A bird has a full 360 degree field of view (it can see in all directions without needing to turn its head). By default the camera is set to 90*. To get the “fish-bowl” effect, where close objects appear very large, this number can be increased. To get an effective orthogonal camera, this number can be set very low. Note that changing this value will also have the apparent effect of zooming the camera in or out, so tapping space to reset the camera view is recommended for large changes.
The effects tab allows for altering the appearance of the 3D output data, without changing the data itself. Current effects are anaglyphic 3D (colour-based 3D glasses), and visual clipping.
The current programs state can be saved to an “XML” state file for later analysis4 . Note that opening an existing program state file will erase your current state. If you wish to merge the two states together into a single analysis, use the “merge” option. Note that as this file references, but does not contain, the data files needed for the analysis, this file cannot be moved between computers and expected to “just work”. However, to overcome this, the program provides the ability to export an analysis “package”, which contains all the data necessary to move these files between computers with ease, regardless of platform. This feature is explained in the “Export” section.
The program has an undo feature which can be used to abort the last changes to the filter tree. Note that for memory reasons, the results of the computation are not stored, and will need to be recomputed. Note that there is also a “redo” function, which allows for undone changes to be restored.
The raw data pane may be used to obtain the raw XY data used to generate the plots. This can either by copied and pasted, or alternately saved to file.
Plots, images, ion data and animations may be exported from the program. The output format for 3D images is the “Portable Network Graphic (PNG)” format; these are supported by almost all image viewers.
For plots, you may save in either (Scalable Vector Graphic (SVG)) or “PNG” forms. Note that due to the nature of the SVG files, no resolution is needed, and the image can be reproduced at any scale. Furthermore the SVG can be used later to generate PNG images at the required size for output (We recommend the program Inkscape). Alternately saving as PNG can be done, and you will be prompted for the desired image size.
Exporting Ion data can be done in several ways; you may export only the visible ions, or alternately, you may export only a subset (for example one or two ranges) of the data, depending upon the filter that the data emerged from (i.e. per leaf filter). The output format will be in Big-endian “POS” format, as detailed in the Appendix, Section 8.2.4.
Modified range files may be exported in whole. Currently the only supported export format is the oak-ridge “RNG” format
Using simple animations of the 3D data can be constructed, where the current camera is orbited 360 degrees around its target location. The result is saved as an image sequence, which can be converted into an AVI using programs such as ImageJ, or ffmpeg to convert the constructed image sequence into a video file.
More complex, filter based animations are covered in more detail in 4.7.7
Finally one can export the entire analysis state, including all required data using the export analysis package option. This will create a folder which contains all the files needed to reproduce the current program state elsewhere. Note that this imports all referenced data files, so the package can become quite large, but should be fully portable to any other system by simply copying the created folder. Inside the folder, the program state will be stored as a state file, and can be accessed by simply opening this state file.
The ranging dialog allows for the complete editing of range files within the program. The range files can be arbitrarily modified, as desired. To access the range dialog, this can be obtained from the Edit-¿Range menu. However, this is only accessible if there are range and spectra available from within the range tree. Note that, at this time (0.0.15), the altered ranges will not be persistent between 3Depict sessions. An initial range file that can be loaded into the filter tree is required at this time.
The dialog is split in two, with a tab panel on the left and a spectrum on the right. If there are several spectra that are rangeable, the spectrum can be selected from the tab panel. Once selected, any existing ranges can be moved and interacted with using the right hand view. Unlike the ranging area, the ranges shown in the spectrum can be moved arbitrarily - i.e., they may overlap, or otherwise be moved past one another. This allows for completely unconstrained editing of the spectrum. Clashes will be shown with a red marker at the top of the spectrum (Figure 15), and must be resolved before the changes can be committed.
New species and ranges can be added using the Add/Remove buttons in the “Ranges” tab. First select the grid you wish to edit, then add the new range. Note that if the range is not fully specified, it will be highlighted in the grid - you must set each field in the grid prior to use.
From the overlay tab, custom “molecular” combinations can be shown on the plot as stick markers, each stick’s amplitude shows the natural abundance for the predicted isotopes5 . To specify a multiple ion, you need to provide the chemical formula in the tab. In Figure 16, the species “TiO” is shown. Similarly, ions such as “TiO2” (TiO2) and “Ti2O” (Ti2O)could be displayed. Note that the overlay to be specified is case sensitive, e.g. “PB2” (hypothetically, Phosphorous diboride) is different to “Pb2” (Lead-2 complex), and thus only the appropriate case will be accepted by the program. Due to the exponential (and thus highly computationally costly) nature of large fragments, only fragments with 10 components will be accepted (e.g. C20 will be rejected, as with 2 C species, there are 220 solutions).
The program will generate an autosave file periodically. If the program crashes, it will look for an autosave file and prompt you to restore it. Note that only the program settings are saved, not the intermediate data, so recomputation will be necessary. If the autosave fails to load, then the autosave file will be archived in your 3Depict configuration folder; in this case, please consider sending the failed file to the developers. For configuration locations, see the paths Section 8.1
As of 3Depict 0.0.12, it is now possible to automate the modify filter-refresh cycle. Specifically this allows for the animation of any property in any filter in the current filter tree. For example, if one wished to create an animation of a slice through of a POS file, one could create a clip filter, then interpolate the 3D point property to move the clip object automatically between animation frames. Note that the output of animation is not restricted to images only, it can output any data that can normally be exported.
In the Figure 17 the main window for animation can be seen, here it is possible to select the filters and properties that are to be animated. Depending upon the type of property selected (e.g. number, colour, string, multiple choice), the selection dialog shown will be different. To select the property that is to be animated, first one must select the filter, similar to how this is done in the main window. Once this is done, the properties currently set for that filter will be used as the default properties for the animation. To change the property, simply double click the property listing on th desired entry. Depending upon the entry type of property selected, as previously mentioned, you will be shown a differing dialog. For example, here the numerical property has been selected, and the dialog for setting animation parameters is displayed (Figure 18).
By setting the start and end frame of the property, as well as the values desired at the start and the end, then the property of the filter will be changed during the animation. Any properties not listed in the grid will remain at their current values. Conflicting values are not allowed, for example, specifying the same property to have two different values at the same time. Such errors will be displayed in the filter view, as seen in Figure 19.
The properties selected for a filter will be linearly interpolated from start to finish, so, for example, setting a property at frame 1 to “1”, and frame 10 to “10”, each frame between will change as 1,2,3…,9,10. This can be done for multiple properties at any one time. Similarly for colours, the interpolation will be done linearly on the colour’s red/green/blue value, so, similarly, colours can also be animated. Whilst linear interpolation can be done for colours, points and digit values, it is not possible to do this for so-called “strings” (text values); these values must be treated specially, as shown in Figure 20.
In this case either each string must be individually specified, as shown in the figure, or alternately, one must supply a text file, with one line per string to be used as input — this can be selected via the “open” button.
Once the desired properties have been set in the filter view, then one proceeds to the frame view in order to review the animation by examining the properties that will be obtained in each filter, frame-by-frame. If you are familiar with this dialog, then it is easy to simply examine this quickly to ensure that the properties that were intended have been obtained. Now, having ensured that this is the case, it can be seen that the ”OK” button cannot be pressed at this time. Firstly, the desired outputs, and the directory that they will be sent to during the animation process must be specified. Once the output directory is set, the desired outputs, normally either images, or points are to be selected as required. The checkbox, if set, will cause only the changes that actually alter the computation internal computation to be saved. As an example, if one was to animate the “Load Limit” property of the “Pos Data” filter, and the file to be opened was only 1 MB, but the animation proceeded as 0.5, 0.6…, 1.0,1.1…, when the sampling value exceeds 1 MB, there is no effect on the computation, thus the program will not save data on frames that do not alter the output.
Different data can propagate through the filtering system before it is seen in the 3D view. The currently available types are ions, plots, range, voxels and drawable object types. Although these are used internally by the program, understanding the type system may enable more advanced use of the program. If you are not interested in this, skip to the next section.
Each ion represents a point in space, which has a value type associated with the point. For example, one might consider a point in a dataset where positions represent atomic positions, and the value is the measured atomic mass. Ions are grouped together by different filters, and each group may be represented with a unique colour and size.
Plots can be passed between filters to allow for a 2D graphical representation of whatever it is that the filter computes. Plots are a X-Y paired set of scalar values, which are finally given a visual representation as a plot. Plots have a title, and a label, and may represented either on a linear scale, or a logarithmic one.
This is a special datatype which propagates information through the filter tree. The data represents non-overlapping regions of the value space which are to be tagged as belonging to a certain group. This data type has no actual output into the 3D scene, but can alter the manner in which “downstream” filters process incoming information. For example, if a profile filter is used after a range, it will split up its measurements into a per-tag “range” section.
Voxels is shorthand for “volume pixel” and is a rectilinear region of space, divided up into an equally spaced rectangular grid. Voxels can currently be represented by a point cloud, where each point has a given colour and transparency, or by a triangulated surface (an iso-surface) which represents the contouring surface for a given scalar value. Experimental support for volume rendering is also available.
3D primitives can be injected into the data stream to assist in the final representation of the scene. Items such as spheres, lines triangles or text can be placed in the final scene.
In this section, the detailed behaviour of the various filters available in 3Depict is outlined. Recall that each filter interacts with other filters and the visualisation environment by generation and propagation of various filter types. At the most abstract level, there are three ways that filters can interact with the data – the list given below provides a may (optional) or will (guaranteed) output.
This section describes each filter in turn, the fundamentals of the internal computation, and provides a table describing which datastreams are emitted, used or blocked during the filter’s refresh cycle.
The data load filter injects 3D point+value data into the analysis tree. Points are loaded from a file by one of several different methods. By default, random data is selected from the file. This filter can be created using the “load” function from the file menu. Note that the default settings will only load a random subset of the data in order to speed analysis. If you require all data to be loaded, then you will need to alter the filter settings.
Information on acceptable data file formats is provided in the Appendix, in Sections 8.2.4 and 8.2.5. This filter is a “base” filter, meaning that it can be used as a base node in the filter tree.
| Stream | Emit | Use | Block |
| Ion | Yes | No | No |
| Plot | No | No | No |
| Drawable | No | No | No |
| Range | No | No | No |
| Voxel | No | No | No |
This loads data from either a raw binary file (32 bit floating point), or alternately an ASCII formatted (legacy) VTK file, using a structured grid [?].
Randomly samples ions from the input stream. Can operate either to generate a fixed number at the output, or to take a fixed percentage of the input. If range information is provided, this can be done on a per-species level.
| Stream | Emit | Use | Block |
| Ion | Yes | No | Yes |
| Plot | No | No | No |
| Drawable | No | No | No |
| Range | No | If available | No |
| Voxel | No | No | No |
This filter allows for the computation of ion counts in any input streams, as well as volume estimation. If a range stream is present in its input, (i.e. a Ranging is a parent of this filter) then the filter will perform per-species computation of the value.
| Stream | Emit | Use | Block |
| Ion | No | Yes | Yes |
| Plot | No | No | Yes |
| Drawable | No | No | Yes |
| Range | No | Yes | Yes |
| Voxel | No | No | Yes |
This allows for the cropping and segregation of ions in 3D space by their scalar values.
Each range loaded from the file may be enabled, either at the ion level (groups of ranges) or at the range level. The range values may be altered; however these may not overlap at any time. Note that these can be edited graphically (to some extent) if used in a mass spectrum. At time of writing, 3Depict cannot be used to generate range files, only write them.
| Stream | Emit | Use | Block |
| Ion | Yes | Yes | Yes |
| Plot | No | No | No |
| Drawable | No | No | No |
| Range | No | No | Yes |
| Voxel | No | No | No |
The bounding box creates a 3D box surrounding any point data in the input stream. The box uses relative coordinates, and has a specifiable font size, colour and line thickness. Several styles of bounding box may be chosen from a predefined list.
| Stream | Emit | Use | Block |
| Ion | No | Maybe | No |
| Plot | No | No | No |
| Drawable | Maybe | No | No |
| Range | No | No | No |
| Voxel | No | No | No |
Note that if the bounding box option “visibility” is set to false, then no drawable item (i.e. the bounding box) will be emitted.
This filter allows for the rejection of data that does not lie within some given boundary. Possible boundaries are plane, sphere and cylinder. For example, if the sphere mode is set, ions within the sphere will be kept and propagated. Ions outside the sphere boundary will be dropped. The clipping object can be placed in 3D by dragging the in-scene object around. Note that holding down Ctrl and shift whilst dragging alter the plane of motion (in-screen, across screen etc).
| Stream | Emit | Use | Block |
| Ion | Yes | Yes | Yes |
| Plot | No | No | No |
| Drawable | No | Maybe | No |
| Range | No | No | No |
| Voxel | No | No | No |
If the drawing primitive is set to be shown, then a drawable stream will be emitted from the filter.
This will generate a histogram of the “value” of ions passing through the filter. Note that no output other than the histogram is generated. Plots can be assigned a colour, set to logarithmic or non-logarithmic mode, or restricted to only cover a specific region. The plot title is taken from the filter name, some limited LATEXis supported (note that the “\” symbol is a special LATEXcommand; you may need to use “\\” to represent a single “\” in the title), for example to type “My Spectrum A\B” you would actually name the filter “My Spectrum A\\B”.
| Stream | Emit | Use | Block |
| Ion | No | Yes | Yes |
| Plot | Yes | No | Yes |
| Drawable | No | No | Yes |
| Range | No | No | Yes |
| Voxel | No | No | Yes |
The profile filter conducts a density or “compositional” analysis of a given sub-region of 3D space. The action of the profile filter depends upon whether the incoming ions have been “ranged”. If not, then the profile filter generates a density profile of the ions inside a cylindrical volume by count, which is visible in the 3D view. If the ions have been ranged, then the frequencies are on a per-species basis. Properties
| Stream | Emit | Use | Block |
| Ion | No | Yes | No |
| Plot | Yes | No | No |
| Drawable | Maybe | No | No |
| Range | No | If available | No |
| Voxel | No | No | No |
Drawable will be emitted if the “Show Primitive” option is selected.
This filter conducts spatially oriented data analysis of incoming ions, and reassigns the ‘value’ component of the ion data. The nature of the reassignment depends upon the selected algorithm and the incoming data itself. Note that the exact values computed by the spatial algorithms may be affected by subsampling; however trends are usually unaffected, provided the number of incoming data elements is sufficiently large.
Local density and density filtering algorithms are relatively simple, and mostly are self-contained concepts. This can be used to identify the local density in your dataset, which in the case of APT, originates due to limitations in the technique. For the Radial Distribution Function (RDF) algorithm , this can be used to examine local correlations between points, which may or may not exist in your dataset. The RDF technique is covered in several standard textbooks on APT [?].
Radial Distribution: The radial distribution function counts pair-pair distances between species. This can be used to find local tests whereby certain point types are more likely to be present at given distances from one another. The distribution can be normalised in the software, to produce a suitable plot.
Axial Distribution: Axial distribution functions are covered in technical literature where they find use in APT and be referred to via a number of differing names, such as “SDM”s [?], atom-vicinity [?], or directional pair-correlation functions. The axial distribution function implementation in 3Depict, allows for users to select and drag out the region to be analysed, with the axis of the cylinder providing both the cropping orientation and the axial direction in which to perform the calculation.
Binomial: The binomial distribution function can be used to test for randomness in the spatial distribution of the points. The program computes a “p” value, which is the probability that the observed data was drawn from a randomly distributed (at the scale of the analysis) set of values on fixed data points. Grouping is performed by a grid-extrusion algorithm, which assigns each set of points in the dataset to a given bin, and thus a given count in the output histogram. The output histogram shows number of occurrences of the observation that a bin contains a given number of counts. This method (or any statistical test) cannot prove randomness, only non-randomness, which may come from many sources. One must be careful when using this mode (or any statistical test), and careful reading of the available literature is recommended. 3Depict does not implement the two-pass method of Moody for computing grid sizes at this time (July, 2016), but rather performs only the first pass.
Point em/re-placement : This function allows for merging points in the current data, A from another file, B. There are several operating modes, “subtract”,“intersect” and “union”. In the subtract mode, points which have a matching element in B will be removed. In intersect mode, only points that have a matching point in the file will be retained - the value to be assigned to the point is taken from B. Similarly, in union mode, both A and B are loaded, but overlapping points in A will be removed and the value taken from B. Only POS files are supported in this mode. Text files cannot be used at this time. The match tolerance parameter controls how close the match must be to eliminate points.
Local Chemistry: This mode computes the local chemsitrysurrounding each atom, by identifying nearest neighbour points species and then using the NN points to assign a “chemistry” value (this could be a concentration, depending how parameters are set) for that atom, similar to the calculations of Chen [?] and Stephenson [?] (Open access). To perform this calculation, there are two types of points “Source” and “target”, and there are two sub-types of “target” as given below:
The value of the concentration is determined using the equation C = 
× 100, where C is concentration in
percent. I.e., the concentration is the number of numerator points divided by the number of numerator and
denominator points. Note that all points that are zero distance from the source point skipped for
the calculation of N and D. If ||D|| is zero, then C is undefined according to the previous equation,
and in this case C is set to -1. In nearest-neighbour mode, the program will keep searching for more
neighbours to complete the computation. If insufficient neighbours can be found, then C is set to
-1.
As a consequence of the definition of numerator and denominator, if these selections are the same, then ||N|| == ||D||, and all ions will be end up being assigned a concentration of 100%
| Stream | Emit | Use | Block |
| Ion | Maybe | Maybe | No |
| Plot | Maybe | No | No |
| Drawable | No | No | No |
| Range | No | Maybe | No |
| Voxel | No | No | No |
Ion, plot and range emit and usage patterns are dependant upon the selected algorithm. Local density and density filtering do not emit plots, and density filtering does allow range propagation.
This mode performs its calculations for concentration in the same manner as local concentration. The primary difference between this filter and the local concentration filter is the output. The output for local concentration alters the value on each ion to reflect concentration. In the filtering mode however, the ions are removed from the output based upon their calculated concentration value.
| Stream | Emit | Use | Block |
| Ion | Maybe | Maybe | No |
| Plot | Maybe | No | No |
| Drawable | No | No | No |
| Range | No | Maybe | No |
| Voxel | No | No | No |
The cluster analysis filter is designed to aid in the detection and analysis of spatial clustering in segregated data. Cluster analyses are used to determine the extent of non-random spatial relationships between dataset members. The cluster analysis filter must be preceded by a “ranging” filter in order to allow for identification of different value types in the dataset
The method works by identifying two data types – “core” and “bulk” members of the dataset (in APT these are usually called “solute” and “matrix”). The program attempts to determine adjacencies between core elements, and to group them together, extracting them from the bulk of the dataset. To do this, the filter uses the scalar value associated with each point to classify it. Range data (which must be present as a parent filter), is used to identify regions of value to classify value regions. Each of these regions then can be selected to belong to either the “core” group or the bulk group – but not both.
The clustering algorithm implemented in 3Depict is a modification of the clustering algorithm outlined in Stephenson et al[?], and to a lesser extent Hyde et al [?] and Vaumousse and Cerezo [?]6 .
Parameter Description:
Algorithm Description; each of these is conducted in sequence to generate the final clustered output.
Note that there are more steps listed in the filter progress due to the need to generate data query structures.
Several post-processing options are available as part of the filter. The size distribution (number of items) can be computed, as can the composition. A frequency table is generated and printed to the program console. Note that the “count bulk” parameter specifies whether to include points classified as “bulk” in these frequency and chemistry tables or not.
Whilst much effort has been placed into optimisation of the clustering algorithm, the query itself is quite slow. The clustering algorithm is best operated on a small region of data to optimise the parameters prior to applying the algorithm to the full dataset.
| Stream | Emit | Use | Block |
| Ion | Yes | Yes | Yes |
| Plot | Maybe | No | No |
| Drawable | No | No | No |
| Range | No | Yes | No |
| Voxel | No | No | No |
This allows the program to run external commands on the system in order to link into other programs. Note: Loading a state file with this filter will result in the user being prompted to the existence of “potentially hazardous elements” in the filter tree, and will give the user the option of removing them. If you are presented with this warning you are highly recommended to discard these elements unless you know better, as it is possible for arbitrary computer programs to be executed if you accept these elements. Short example programs for transferring data into and out of 3Depict are given in the appendix, Section 8.3.
Command syntax: The syntax for specifying the program command uses % as the escape character. If you wish to pass a single % to the command line, you can use %%. %i will be substituted with the first pos file’s name, repeated uses will use the second, third and so on-th pos file name. If there are not enough incoming ion streams to be converted to pos files, then the filter will report an error. You can use %I to substitute all pos files (space separated) to the command line at once. Similarly %p and %P will substitute for plots. Unrecognised % sequences will be considered an error.
Prior to program execution: Ion data coming into this filter will be saved in the folder “inPos” inside the specified working directory, with the prefix “pos”, in the pos format (Section 8.2.4). Plots will be saved as tab separated files with the prefix “xy”. If there is no input to the filter, and thus no files, the program will not be run. By default, the program that is executed will have these files passed as arguments to the function, appended to the output if no % syntax is used.
At this point, the target command will be run. 3Depict will halt at this point, and await the completion of the underlying program.
After program execution: Once the program is run, any .pos files (i.e. any files matching ‘*.pos’) in the working directory will be loaded back as ion streams. Similarly any ‘*.xy’ files will also be loaded. .xy files should be ASCII files, and should have a multiple of 2 columns (one for x, one for y) separated by a valid delimiter. The x and y column lengths must also match for each x-y pair. Valid delimiters are tab, comma and space. At time of writing, there is currently no way to specify the plot colour or style. The x-y values will, by default be connected with line, thus a single value will not be clearly visible
Input files will be semi-randomly named to mitigate “collision” problems in the case that multiple instances of 3Depict are being used.
| Stream | Emit | Use | Block |
| Ion | Maybe | Yes | Yes |
| Plot | Maybe | Yes | Yes |
| Drawable | No | No | Yes |
| Range | No | No | Yes |
| Voxel | No | No | Yes |
The annotation filter allows for insertion of 3D annotations into the 3D scene. The annotations that are allowable include linear and angular measurements, as well as textual markers and text+line pointing markers. Each annotation mode has an associated colour which is specifiable.
Text Displays a 3D textual marker in the 3D scene, using a fixed font.
Arrow, Arrow with Text Displays an arrow from one position to the other. This can be directly interacted with in the 3D scene. The Arrow+Text mode allows for a specifiable arrow and associated text point in 3D space in one.
Angle Measurement Displays a 3D widget which can be used to mark angular relations in 3D space.
| Stream | Emit | Use | Block |
| Ion | No | No | No |
| Plot | No | No | No |
| Drawable | Yes | No | No |
| Range | No | No | No |
| Voxel | No | No | No |
This filter discretises space into a series of 3D cubed regions, known as “Voxels” (Volume Pixels). Voxels can be used to alter a point cloud into a discrete volume of counts associated with each region. For example, the number of points inside each cube can be used as the counting metric, the number of points of a certain ion type, or the ratio of the number of points within a given region. Voxels can be divided up into either fixed-width bins (a fixed step size in space), or it can be divided up into a fixed number of bins. These do not need to be the same size in each dimension.
If the step size is too small, or the number of bins are too high, you may find computation is very slow, or you will run out of ram. This is because the requirements for ram are proportional to the cube of the number of counts.
At this time, there are the following representations for Voxels:
If a range file is present in the input, this can be used to perform per-range computations, such as ratio voxelisation specific to particular species. Figure 22 shows the different representations on the same input data.
Note that the limits of the values associated with the voxels will be printed on the console after the filter has refreshed – this can be used, for example, to set the limits for isosurface representation.
There are different normalisation options available for the voxel filter. If no range data is present, you can specify either no normalisation, or normalising by each voxel’s volume (this is a constant across the dataset). If range data is present, then you can additionally noramlise by total counts to get composition, or you can specify a custom numerator and denominator.
The data computed can be “filtered” to obtain, for example, a smoother visualisation of the dataset. Note that by smoothing the data, you are spreading out information, and you lose the ability to see small things, however fluctuations due to noise are damped. The spread of the guassian blur can be specified, as well as the size of the local neigbourhood to use when computing the filter.
It is also possible to “mask” the data, such that any data with fewer than a given number of counts (if using a ratio, this is the number of counts in the denominator) is replaced either with the minimum or the maximum value from the remaining volume. This can be used to, e.g. suppress spurious features that occur simply because there are low counts. If the “Mask to Max” option is selected, then the masked values will be replaced with the maximum in the dataset rather than the minimum. This can have the effect of “inverting” the isosurface volume selected
| Stream | Emit | Use | Block |
| Ion | No | Yes | Yes |
| Plot | No | No | Yes |
| Drawable | Yes | No | No |
| Range | No | Yes | No |
| Voxel | Yes | No | Yes |
This filter allows for the association of a particular colour to an ion, based upon the value of the ion, and the desired colour scheme. By selecting a start and end value for the colour scheme, Points can be given a colour that interpolates between these two values. The ion’s value (as, for example, visible in a spectrum histogram) is used to set the colour for that point.
| Stream | Emit | Use | Block |
| Ion | Yes | Yes | Yes |
| Plot | No | No | No |
| Drawable | Yes | No | No |
| Range | No | No | No |
| Voxel | No | No | No |
This filter allows for the transformation of the XYZ or value of a given point, based upon the chosen filtering algorithm. The available algorithms for transformation of a point are as given:
| Stream | Emit | Use | Block |
| Ion | Yes | Yes | Yes |
| Plot | No | No | No |
| Drawable | Maybe | No | No |
| Range | No | No | No |
| Voxel | No | No | No |
A drawable will only be emitted if the “show marker” option is selected.
This document is licenced under the Creative Commons 3.0 ShareAlike licence. http://creativecommons.org/licenses/by-_sa/3.0/
3Depict uses one of several different paths for storing configuration and autosave information, depending upon your host operating system.
The state file is an XML file, which is generated by 3Depict. XML is short for eXtensible Markup Language, and describes a basic format for data layout. In XML files, the file consists of elements, attributes and text. Each element is marked by the use of angle brackets, as in “<element>”. Each element must have a start and an end marker, for example <element> is the start marker, and must be followed by its end marker </element>. These elements can be nested or in sequence, but cannot be mixed (it is wrong to say <element1><element2></element1></element2>).
A full description of the XML language is beyond the scope of this document, however many resources can be found online to explain the concept. The extensible bit means that 3Depict can define its own elements. Hence the exact format is subject to change from version to version.7
This is due to the rapidly changing nature of the filter properties. Unfortunately the most up-to-date documentation for the file format is the source code itself. For those who may be inclined to try to emulate this, you may wish to look at the VisController::saveState routine, and the Filter::writeState routines.
However, in general the file is divided into several sections. Below is an example file.
The state consists of the program version, to check that the program can actually interpret the file, a background colour, and the filter tree.
Optional elements which are not shown in this example include the stash data, and the camera information (here there is only the default camera).
The filter tree is shown, with a pos load filter as the top level element, which has a child element of iondownsample. Note that the attributes of each element are dependant upon the filter. Again due to the rapidly changing nature of the program, this is subject to change.
Remap files are an XML formatted file that defines how one ionic species can be broken up into other species. Entries in the remap file are given a name, and the daughter ions which comprise these. For example, one can convert the species TiO2 into 1 Ti and 2 O ions, by defining an entry in the remap file. Filters which allow you to provide a remap file can utilise this information e.g. when determining compositional information. Each entry must be unique, however what it decomposes to need not be unqiue.
An example remap file is given below:
3Depict’s interpretation of the Oak-Ridge format for range files is given below. The original specification is available in the book Miller, Atom probe: Analysis at the atomic scale, (Kluwer Academic/Plenum Publishers, ISBN 0306464152). Additional information on the format is given by the PoSaP program, which to the author’s knowledge is not online. Unfortunately, the specification given for the file is weakly stated, and is open to different interpretations. 3Depict is designed to be as resilient as possible to variations that have been encountered, however it may be that there are alternate interpretations with which the authors are not familiar, and the code is thus unable to interpret.
A simple example file is given below, and is nominally in the ASCII 1 byte per character format. The original specification, to the authors knowledge, predates the UTF-8 and extended codepage support for non English languages. Thus non-English languages are not part of the file format - each should use the “C” locale for reading and writing, to avoid localisation concerns.
The first line consists of two unsigned integers, separated by a space. The first integer is the number of unique ion types, and the second is the number of ranges. The next lines are taken as pairs. The first entry in the pair is the name of the “ion”. The next entry consists of four parts. The first entry is a space terminated string, and is the shorthand name for the element. The next three elements are floating point values in the range of [0,1], and are the colour of the ions that are ranged, with each element being the red, green and blue component in turn (i.e. cubic RGB space). The final string is, to the authors’ knowledge, unused, and is ignored by 3Depict.
This is repeated for each element pair, as specified by the first integer in the file. Each entry must be uniquely named, both in short and long names.
The next line can nominally be ignored, however it should contain the dash character from positions 1 to 13, followed by a space separated list (with leading space) of the short names, as specified above. Sequence positions are not obtained from the dash list, but rather from the order they appear in the file.
Following this is a 2D table (space separated). The first column appears vestigial. The second and third columns contain the start and end “range” values for each ion. Note that these do not have to be in the same sequence as the original specification. These range values must be non-overlapping, and can be any 32 bit floating point number (other than NaN).
The next columns are the range table, and specify which ions the range corresponds.
In the 3Depict implementation, the table should have only entries of 0 or 1, and the row (from column 3) should to exactly 1. Files where this is not the case may be accepted, however the exact interpretation for non 0/1 entries is unclear, and not specified in the file, so will be essentially treated as either a 0 or 1 value.
A more complex example is given below.
Note that3Depict guarantees to be able to read its own range files, and will do its best to read files generated by any major external program (within reason). If you have a file that you believe should be accepted, please contact the author.
This file is a four-field fixed width record file, with an integer number of entries. The file is uncompressed raw 32 bit IEEE754 floating point data, and can be loaded using most languages relatively easily. Note that the order of the floating point numbers “endian-ness” is fixed as big-endian. The floating point values are X,Y,Z and an arbitrary scalar value. The file may not contain invalid (Not-a-Number “NaN”) values. 3Depict will accept files with different numbers of records (eg XYZ only, or XYZMI (where I is ignored)), but this must be manually specified in the DataLoadFilter.
The text files that are accepted by 3Depict must be ASCII formatted, and consist of at least four columns of data, separated by an acceptable delimiter. The accepted delimiters are currently tab, space and comma. The numeric format must be in the English locale, i.e. with a period used as the decimal separator, consisting of the digits 0–9 and the + and - symbols.
Each file may have a contiguous header that does not consist of this format, however if ANY portion of the header is interpretable as per the above, this will be considered to be the end of the header, and the remainder of the file is the file body. All lines in the file body must be interpretable as per the above.
Note that due to the need to do multiple passes over the text file to interpret it, and the need to do string to binary conversions, this will be considerably slower than using a POS formatted file for large inputs.
Voxel data is exported in two formats. Firslty, we export files as “raw” binary files, as they are represented in-memory on a PC. In this case, each voxel stores data as floating point values, which represent the contents of the voxel. These are stored in either row or column-major order (not specified), and 32-bit IEEE754 floating point values, in native-endian order (this normally corresponds to little-endian). Tools such as ImageJ should be able to open such files.
When exporting in VTK mode, this is a single file in the so-called “legacy” VTK format [?] in ASCII mode. This format is developed by Kitware/Sandia national laboratories, and the full specifications are online. The format is supported by 3D visualisation software such as “paraview” [?], and “Slicer”, both which have advanced volume analysis capabilities.
The “external program” filter can be used to transfer data on-the-fly between 3Depict and a separate program, allowing for an extension of the capabilities of 3Depict, without requiring direct modification of 3Depict itself. This is targeted at advanced users who wish to connect other programs to 3Depict, as part of their analysis toolchain. Here short, simple example programs are given in several languages. Specifically, we provide examples for Scilab, Python, Bash and C/C++ - representing an accessible breadth of differing programming languages. In each language the fundamental principles of loading and returning data to and from 3Depict is the same.
Each example will load an input file, generated from 3Depict, optionally alter the data, and then return the modified data back to 3Depict. The examples are for instructive purposes only, and do not reflect the optimal implementation of the specific task, in order to simplify the presented program. Not all features of the external program filter are presented in this Appendix. For the full documentation on the filter, see Section 5.2.5.
Files for the sample programs can be generated from the following inline examples, or alternately, can be downloaded form their respective URLs
This example uses the computational package Scilab can be used with 3Depict. Scilab is available online, at http://scilab.org and is a general numerical computing package. You can adapt this for Octave 8 or Matlab (if you have access to that). For large data volumes, we recommend using python or better C/C++. General purposes numerical packages are ill suited to large datasets, due to typically poor memory management capabilities.
In this example, the script simply opens a file, moves the point cloud by -1,-1,-1, then saves the output. The output, and input (due to the Bounding box filter) are both visible in Figure 23 - note the offset induced by the script. During refresh of the filter tree, the Scilab interface appears, performs its computation, and then exits, as instructed by the script. Any desired computation could be performed at this stage - exiting Scilab can also be done at any time by the user by removing the final exit instruction. This procedure may be useful if interactive querying of the dataset was desired.
Note that as shown in the figure Scilab is called using its ‘-args’ parameter, which avoids Scilab from attempting to parse the arguments you wish to pass to the script as its own. Further note that this example will not work if any filename or directory (including the working directory) contains a space, due to this behaviour. In this case, 3Depict was launched from its own folder, which does not contain a space.
This example demonstrates using Python to interact with 3Depict. The following example does very little - it simply loads all input pos files (due to the %I in the program invocation), and merges the contents. The results of the computation are shown in Figure 24.
The following trivial program shows how 3Depict can be used send data to and from Bourne Again SHell (BASH) programs. 3Depict was launched with one pos file, and an external program filter, as shown in Figure 25. The script used in the test, named “test.sh” and placed in the specified working directory (see figure), is given below. The object of the script is only to demonstrate that the script can be used to perform arbitrary actions, not to perform any actual data manipulations.
The output from running the refresh cycle is given, as it appears on the program console (to replicate this mac/windows users may need to redirect the output to a file in order to see the output text (this can be done in bash), or mac users may launch 3Depict from terminal.app).
Firstly, note that the same file is written to each time - 3Depict does not delete the “script-output-3Depict-input.pos”, so if this is named differently between refreshes, multiple pos files would be generated and 3Depict would load them all. Finally, the statement exit 0 is used to ensure that 3Depict knows that the program terminated successfully. Recall that returning a nonzero value will inform 3Depict that some error occurred during processing, and thus it will abort further data processing.
The output from the program will be similar to the following text.
For C/C++, an example is given. The example here is somewhat more complex than the rest, as in this case, we do not simply treat the data as a series of bytes, but we additionally perform the data transformation steps required to get it into a usable form (i.e. as a list of correctly ordered bytes in memory, in a useful variable). This was not done for the previous examples.
Note that as C/C++ are compiled languages, it is necessary to be able to be able to generate a binary (executable) version of the program - this procedure is not described here, but users are encouraged to be comfortable with this process before attempting to implement the following examples for themselves. The exact procedure for doing this is outside of the scope for this document - you will require a compiler, such as gcc’s g++ compiler, which you will most likely want to install from some form of package management system, such as the APT, yum or zypper systems on linux, Xcode or Macports on Mac OSX, or Cygwin or tdm-gcc/msys under windows. Being able to compile a program file to produce an executable binary (under windows ‘EXE’) that consists only of int main() {} is a definite prerequisite.
For this program, once your compiler is installed, and assuming you use gcc, the normal procedure is to place the following code into a text file called example.cpp, then run g++ example.cpp -Wall -o example, to produce the binary. You must be execute that command in the same folder as the example.cpp file is located.
This produces a binary file, called “example” under linux/OSX or “example.exe” under windows, now setting up 3Depict as per Figure 26, several ranged ionstreams are passed to the program, which are merged into a single file. This single file is detected automatically by 3Depict, as it is a file ending in “.pos”, and is located in the working directory - it is thus assumed to be loadable.
As 3Depict is an open source program, you are free to modify it, or to extract useful bits subject to the licence agreement (See Section 2.1.1). You will need a knowledge of C++ in order to reasonably understand the components of the program itself. A knowledge of OpenGL and wxWidgets is desirable, but you could pick this up as you went along, and don’t really need it for many parts of the program.
This section of the manual is the hardest to write, and the most likely to not be applicable to your context, as it depends heavily on the computer system you are trying to use. Nevertheless, this section will attempt to explain how to get yourself set up to build. To modify the program, you must first be able to build the base version of the program from source code. This is by far easiest under a Linux system, as packaging programs can allow you to auto-import all the needed components to build the program.
The program was primarily developed using C++ (gcc), and utilises autotools for the build scripts. Some custom Bourne-again shell (BASH) scripts are used to do side tasks, such as dependency retrieval and compilation and .app package building (for OSX, really). Mercurial is used for version control. The program is developed using a private repository, which is synced up to the Sourceforge repository periodically. My personal tools for development are the VIM editor and the command line. This was primarily developed under a Debian squeeze (testing) system (EEE 901), with some development under OpenSuse. The authors actively maintain the programs’ package for Debian, and this is periodically synchronised to Ubuntu’s package database. The program is also available under the Fedora platform.
The main libraries used for the program are:
Compilation instructions vary from operating system to operating system. In increasing order of complexity to generate a compilation, Linux, mac, and windows versions can be built from source. Instructions for compilation change frequently, and the most up to date version is available on the project website.
If you are running a Debian or Debian derived distribution, all you need to do is to run these commands as an admin user sudo aptitude install build-essential, which will install a compiler and the needed build scripts. Then run sudo apt-get build-dep 3depict, this will install all the needed components to install the program.
Once this is done, you can download the latest source code from the website, unzip it, and then run ./configure && make. This builds the program. You can now modify any of the files, then recompile it simply running make. By examining the options listed by ./configure --help, the configuration of the program can be altered to some extent (e.g., enable/disable debug checks, or computational parallelism).
As 3Depict is open source, it can be modified in the case any error fixes, extensions or other alterations to the program are desired. However, a certain level of prerequisite knowledge is necessary to effectively alter the program. If altering code in 3Depict, You should be familiar with C++, as well as compiling multi-file programs. Depending upon the modification, you may need to have some familiarity with the mathematical problems you need to solve and with libraries used by 3Depict, such as OpenGL or wxWidgets. Instructions on how to compile for the various platforms is given on the website. However, due to the changing nature of these platforms, no guarantees that compilation will be successful can be given, when following these instructions – some debugging of the process may be required.
The internal structure of the program can be more easily seen from the Doxygen documentation, which is listed online, or can be generated from the source files themselves via the Doxygen tool. If you want to have a play around with the code, try getting it to compile first, before trying to change anything. Feel free to drop us a line on the website to ask about the change you want to make, and how it could be most easily achieved in the code.
All you have to do now is to modify the .cpp and .h files to do what you want, this is going to be specific to what you want to do, and thus it is impossible for a “walkthrough” to be reasonably written. This is only here as a guide. To get started, the easiest components to change are probably the filters. These have been written to be as independent of the user interface as possible, so you need to know very little (almost nothing) about OpenGL or wxWidgets to modify them.
Each filter is an object derived from a base class, Filter. To implement a new filter, you have to derive a new class, eg MyFilter, and implement the pure virtual functions to do what you want. To make it accessible from the user interface, you have to add a new entry in comboFilters_choices in the function MainWindowFrame::MainWindowFrame(…), in MainWindowFrame::OnComboFilter(…) in 3Depict.cpp as well as in makeFilter(…). You can probably just copy the relevant bits from a neighbouring filter.
The main purpose of the filter design is is that each filter takes in something, and spits out something - by simply implementing a new filter, a new effect can be achieved. The filter’s job is to convert input FilterStreamData, into some other output. Each filter does all of its work in the ::refresh functions. The easiest examples of this will probably be the ion info and transform filters. They might look a bit daunting, but much of the code is simply there to keep things running as fast as it can, and to provide many options. Each filter can only work with its own properties, and that of the FilterStreamData pointers passed into the refresh function. It can really do anything it wants here. The refresh logic will examine the filter outputs to determine their consistency (both internally, and against the filter data output matrix) when compiled in debugging mode, generating errors on any detected inconsistencies in the output.
To have properties appear up in the left hand panel, you have to implement the getProperties(…) function – try copying one that seems closest to your situation. To have these properties take effect, you need to implement the setProperties(…) function. If you wish to “peek-ahead” at the filters coming into the filter (this is a little advanced, but can sometimes be necessary), you have to implement the initFilter(…) function.
