The NetX I/O application comes in two different forms:

  1. Desktop App (Mac OS and Windows)
  2. CLI Tool (Linux, Mac OS, Windows)

This article will familiarize you with the CLI (command-line interface) version. While the Desktop version makes it easy to set up multiple syncs, along with drag and drop importing, there are cases where you'll want to programmatically drive NetX I/O. For example, the CLI allows you to set up various sync point on a workgroup file server. It runs "headless" (no user interface), and can be easily wrapped in shell scripts and scheduled via cron (or others). The NetX I/O CLI is packages as a Jar file, and is executed with Java.

Installation

This outlines the steps needed to start running the NetX I/O command-line tool. This illustrates setting it up in "sync" mode, but if you want to use the tool in WatchedFolder or any other mode, the steps are almost the same.

Windows  users: NetX I/O requires the Microsoft Visual C++ Redistributable to be installed before it can run. For instructions on how to obtain this package, see https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads.

Step 1: Make sure JDK is installed locally

You can test this on Linux or Mac by typing: "java -version" to check the Java version. Like this:

$ java -version
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
CODE

If the version printed is anything other than "1.8.x", you will need to download a supported version of Java from Oracle: https://www.oracle.com/java/technologies/javase/javase8-archive-downloads.html

Step 2: Download the NetX I/O jar file

You can download the appropriate version (eg "netxio-4.0.xxx.jar") from our Service Portal, found in the /Modules/NetX I/O folder. 

Step 3: Create a work location folder

Create a folder on your workstation for NetX I/O to write files. For the sake of illustration, let's say you're on a Mac, and you create a NetXIO folder inside your home directory:

/Users/me/NetxIO
CODE

Step 4: Copy over the jar file

Copy the netxio-3.0.xxx.jar file into your work location folder. Eg:

/Users/me/NetxIO/netxio-4.0.001.jar
CODE

Step 5: Create your sync folder

The storage location you created in Step 3 is simply for managing administrative files. Now you need to choose or create a new folder that will be the root of all the synced sub-folders and files. Again, to illustrate, we'll add one off the Desktop; here for example:

/Users/me/Desktop/My Project Files
CODE

Really, you can call this anything you like, and it can live anywhere. Just choose something convenient to use. Do NOT nest the work location folder inside of your sync folder.

Step 6: Create (or choose) your sync folder (in NetX)

You can choose any folder that is not a top-level folder. However, please choose wisely: it's better choose a folder whose sub-structure is not very large. You can, but the sync times, bandwidth, and local storage requirements will obviously increase. Make sure that you are choosing a folder that the account you are running NetX under has appropriate access permissions.

For our example, we'll choose this folder in NetX: 

/Projects/Me/Sync Files
CODE

Step 7: Create (or choose) a user (in NetX)

NetX I/O logs into NetX using supplied credentials for a pre-existing user, and actions will be performed on behalf of that user.  It's best to create a separate account for your NetX I/O user. If you were to log in to NetX and then log out as the same user logged in via NetX I/O, it disrupts the processes in I/O. If you're running NetX I/O in Sync mode, you must configure it to run with an Administrator-level user. For Watched Folder mode, a Producer-level user (or higher) is required. You will need to know the username and password for that user you want to log into NetX with.

Step 8: Start NetX I/O

On a Mac, you can open a terminal, and type the following:

java -Xms512M -Xmx1024M -cp "/Users/me/NetxIO/netxio-4.0.001.jar" com.netxposure.external.client.io.NetxIO -url https://acme.netx.net -username me -password xxx -netxSourceFolder "/Projects/Me/Sync Files" -localSourceDir "/Users/me/Desktop/My Project Files" -mode sync -localStorageDir "/Users/me/NetxIO" -useInsecureSSL -loop 15
CODE

Ensure the "cp" directive is placed before the class (com.netxposure.external.client.io.NetxIO), or it will be interpreted as a parameter to the class. Instead, this is intended for Java. Also, this process could potentially use a large amount of RAM (depending on the total sync size; it will need to hold the entire local state map in memory, twice to compare). So in this example the Java heap is set to start with 512MB (-Xms512M) and can utilize up to 1024MB (-Xmx1024M). This is just an example.


Most of this should now be familiar to you, including the library directory, the username, password, folder, directory and local storage. But to formalize this, here's each flag with corresponding details (please refer to the Usage article for more details and options):

ParameterExample ValueDetails
-Xms
512M
This parameter sets the minimum memory for NetX IO. 512MB of RAM is usually sufficient. This parameter MUST come before "com.netxposure.external.client.io.NetxIO".
-Xmx
1024M
This parameter sets the maximum memory for NetX IO. 1024MB of RAM is usually sufficient. This parameter MUST come before "com.netxposure.external.client.io.NetxIO".
-cp 
"/Users/me/NetxIO/lib/*"
This parameter sets the Java classpath and must point to the NetX library directory. This parameter MUST come before "com.netxposure.external.client.io.NetxIO".
-url 
https://acme.netx.net
This is the URL of your NetX server instance.
-username
me
This is the username of your NetX user.
-password
xxx
This is the password of your NetX user.
-netxSourceFolder
 
"/Projects/Me/Sync Files"
This is the folder (or "category") in NetX which files and folders will sync.
-localSourceDir
"/Users/me/Desktop/My Project Files"
This is the local directory where folders and files are synced with the NetX server.
-mode
watchedFolder
This sets NetX I/O into "Watched Folder" mode.
-localStorageDir
"/Users/me/NetxIO"
Local storage for NetX I/O administrative files, including logs, clientId data file, and lostandfound folder. Please see storage capacity note below.
-useInsecureSSL

This is actually not usually recommended. But setting up certificates is out of scope for a quick start guide.
-loop
15
This sets NetX I/O to run the sync process every 15 minutes.


If you start NetX I/O using the terminal, you may want to background that process, because not doing so and closing the terminal will quit NetX I/O. Additionally, you may also want to add this command to your workstation startup items. There are many ways to deploy this tool more robustly, but that's out of scope for this quick start. This article's purpose is simply to get you up and running quickly. 

Be aware of the capacity of the underlying storage when configuring the "localStorageDir" setting. If imports encounter significant errors, failed import files will be copied to the "lost and found" directory under this path. If that storage facility runs out of space, that will lead to secondary (and critical) errors with NetX I/O.

If the user account running I/O does not have appropriate access permissions to the NetX folders it is trying to sync, an error will be received:

[NetX I/O] ERROR	(com.netxposure.external.client.rpc.ApiException: JSON-RPC Error 4003: Object not found (invalid category_id (18)) (null))

Step 9: Start using the sync

Drop folders and/or files into your local sync folder, and/or import new files into NetX. Wait for the loop time, and then see the folders and files replicated on the other side. Enjoy!

Built-in help

You can obtain details on all the NetX I/O by running the CLI tool with the "-help" flag:

java -cp "/netx/8.11/webapps/ROOT/WEB-INF/lib/*" com.netxposure.external.client.io.NetxIO -help
usage: help
 -blind                        For watchedFolder, import, and export
                               modes, specifying 'blind' means the
                               analysis step will be skipped. When
                               enabled, it can be much faster if you are
                               not concerned with avoiding uploading
                               duplicates. This option also allows for
                               versioning of files on the NetX server
                               (watchedFolder and import modes only).
                               Optional.
 -clientId <arg>               Normally NetX IO automatically creates a
                               Client ID, but if you want to control this
                               for any reason (advanced), you can set it
                               with this option. Optional.
 -config <arg>                 Local path to a config file, which can be
                               used in place of other commandline options.
                               This config file must be a properties
                               formatted file (name=value)
 -databaseMap                  Use the database temp map? The default is
                               to use an in-memory map; it's faster, but
                               may require more memory. Optional.
 -debug                        Log debug messages? Optional.
 -deleteLimit <arg>            If you are running NetX IO in 'sync'
                               mode, and if you delete more than a certain
                               number of files in a single sync cycle,
                               NetX IO will automatically halt because
                               it will assume it's a mistake on the local
                               workstation. The default delete limit is
                               '100'. Setting this to zero ('0') disables
                               any deletion limit. Optional.
 -help                         Print help message. Optional.
 -localSourceDir <arg>         Local source directory from which to look
                               for files. Required.
 -localStorageDir <arg>        To enabled 'sync' mode, or to have logs
                               write to a file, you need to provide a
                               local storage location where files can be
                               written. Required for 'sync' and 'merge'
                               modes.
 -loop <arg>                   Loop allows NetX IO to run continuously.
                               Please set to a number, measured in
                               minutes. NetX IO will pause between scans
                               based on the configured minutes. Zero is
                               the default, in which case the scan only
                               runs once. Optional.
 -mode <arg>                   Options include: 'watchedFolder' [Only send
                               files and folders from client to NetX;
                               delete client files when done, and leave
                               folders.]; 'sync' [This mode compares the
                               client sync map with the NetX map, and
                               reconciles differences between the client
                               and NetX. Supports limited moves and
                               renaming changes, as well as deletes and
                               additions.]; 'merge' [Merge client files
                               and folders with NetX files and folders;
                               nothing will be deleted. This runs the for
                               the first time in sync mode, since there
                               are no client state map at that point.];
                               'import' [Only send files and folders from
                               client to NetX; leave client files and
                               folders when done.]; 'export' [Only send
                               files and folders from NetX to the
                               client.]; 'analyze' [Emails an import
                               report to the calling user's email. No
                               other action is taken; no files or folders
                               will be affected.]. Required.
 -netxSourceFolder <arg>       Remote NetX folder where to store the
                               imports (eg "/Category1/Category2"); if the
                               value is a number, then this is assumed to
                               be the corresponding category ID. Required.
 -noHeartbeat                  If you are running NetX IO in a loop, by
                               default it will set up a heartbeat with the
                               remote server; if that heartbeat stops, it
                               will notify you via email. Use this flag to
                               disable this function. Optional.
 -nonvolatile                  By default, NetX IO will automatically
                               scan the source directory, wait 20 seconds,
                               and then scan again to ensure all the bytes
                               on all the files are static (files are not
                               still being copied). Adding this flag turns
                               off that safety check such that the sync
                               will commence immediately. Optional.
 -password <arg>               Password. Required.
 -replacementCharacter <arg>   If illegal characters are encounter in
                               either files or folders, then use this
                               replacement character (single). Optional.
 -report <arg>                 Set the reporting period. Available values
                               are: 'daily', 'weekly', 'monthly', and
                               'none'. Default is 'weekly'. Optional.
 -timeout <arg>                Advanced. By default, all web service calls
                               timeout after 3 minutes. Only change this
                               if you know what you are doing. Value is
                               measured in minutes. Only non-zero values
                               are valid. Optional.
 -url <arg>                    NetX instance URL (eg
                               "http://acme.netx.net"). Required.
 -useInsecureSSL               If you are connecting with HTTPS and can't
                               register your SSL certificate with NetX
                               Sync, this will disable any certificate
                               verification. Optional.
 -username <arg>               Username (for 'watchedFolder' mode, this
                               must be an Producer-level user; for 'sync'
                               mode, this must be an Administrator).
                               Required.
CODE

Merge and Sync modes are experimental and are not supported for use in mission-critical environments. Use at your own risk.

Running multiple NetX I/O processes on a single workstation

There are cases where you may want to run multiple instances of NetX I/O on a single workstation — and that's perfectly fine. However please ensure that each instance uses its own local storage and sync folders. Do not attempt to run multiple instances of NetX I/O CLI with a shared local storage directory!

If you're using a config file (see details below), and order to identify the various processes running presumably in the background, you can add a -D parameter to java. The "-D" is normally a mechanism for passing name+value pairs to the JVM. However, the JVM is happy to ignore directives it doesn't understand. So you can add something like "clientId=35" to the command-line argument so that if you need to query the process table (speaking specifically of Unix platforms here — Linux and Mac), you can see that argument there. 

If you want to do this, you must add the "-D" flag before the reference to com.netxposure.external.client.io.NetxIO. Eg:

$ java -DclientId=35 -Xms512M -Xmx1024M -cp "/Users/me/NetxIO/netxio-4.0.001.jar" com.netxposure.external.client.io.NetxIO -config "/Users/me/NetxIO/client1.properties"
CODE

Doing this means that if you type this on a Mac:

$ ps ax | grep java
CODE

You might see something like this:

26916 s010  S+     0:03.92 /usr/bin/java -DclientId=1 -Xms512M -Xmx1024M -cp "/Users/me/NetxIO/netxio-4.0.001.jar" com.netxposure.external.client.io.NetxIO -config "/Users/me/NetxIO/client1.properties"
CODE

Using a config file instead of command-line arguments

You can also set all your NetX I/O options within a configuration file. In this scenario, you only set the configuration file (the "-config" option) in the command-line. Like this:

java -Xms512M -Xmx1024M -cp "/Users/me/NetxIO/netxio-4.0.001.jar" com.netxposure.external.client.io.NetxIO -config "/Users/me/NetxIO/netxio.properties"
CODE

Here's what an example configuration file might look like:

url=http://127.0.0.1
username=admin
password=xxx
netxSourceFolder=/My Files
localSourceDir=/Users/me/Desktop/NetxIO/files
localStorageDir=/Users/me/Desktop/NetxIO/store
mode=sync
debug=true
clientId=My Client Example 35
CODE

Note that option names are separated from their corresponding values with an equals sign (=). Also, do not wrap config values with quotes (like you might do on the command-line). Additionally, please note that command-line arguments that require no arguments — like "debug" in this example — can be set with a value of "true".

I/O lock file behavior

What happens with NetX I/O CLI if the last execution is still running when cron tries to execute again? The lock file should stop it from executing concurrently. NetX I/O writes a lock file IF there's a configured "localStorageDir". There, it will write a tmp file with a timestamp. And that will regulate how often another NetX I/O process — configured with the same localStorageDir — can run. For "loop" based tasks, the time-lock is twice as long as the loop. If not, it's 30 mins. This file is then periodically updated throughout the processing of the run. The reason for the timestamp is so that if NetX I/O terminates unexpectedly and the lock file is left there, later runs of other I/O processes can note that the lock file itself is "stale" and proceed regardless.