This is a NetX Labs feature! We couldn’t wait to get your feedback on it, so we are providing you early access, even though there is still room for improvement. Don't hesitate to let us know your thoughts about this feature in the NetX Ideas Portal.

Overview

Integrating NetX with Google Vision allows you to auto-tag image assets based on content that is detected by the Google Vision engine. The following Google Vision functions are supported:

  • Label Detection – Identifies broad categories within an image, such as modes of transportation or animals. The confidence score threshold is 75% (tags with a confidence score of less than 75% are ignored and will not be tagged in NetX).
  • Logo Detection – Identifies common product logos within your images.
  • Landmark Detection – Identifies popular natural and man-made landmarks.
  • Face Detection – Detects multiple faces in a single image, as well as facial attributes which may indicate emotional state as well as identifying headwear. Facial recognition is not supported. 
  • Extract Text — Optical Character Recognition (OCR) enables you to detect text within your images. 

For more information, please refer to Google's official documentation

Setup

Properties

Please note: Correct configuration of the following property is mandatory in order to use this feature. However, if your instance is hosted by NetX rather than on-premise please contact support to help you establish this property. 

PropertyTypeDefaultDescriptionRequires Restart?
external.google.vision.credentialFilePathStringnone

This will be the full file path for the Google Service Credential JSON file. To obtain this file, see the section on creating your Google Service account.

yes

Google Service Account

To use Google Vision integration, you must first create and use a Google account and obtain a service account key: 

  1. Follow along with Google's official set-up instructions. You will only need steps 1-3 under "Set up your project"; you will not need a Cloud Storage bucket.
  2. Next, follow along with Google's steps to create a Service Account JSON key. Begin at "Set up a service account". Once you have a JSON Service Account key file, you are ready to proceed.

NOTE

If you have an on-premise deployment, you'll have to install the JSON credential file on the NetX server and then configure the Google Vision property to reflect the path to that file. If you are using NetX SaaS, then you'll want to work with Support to securely transfer your JSON credential file to NetX, and Support will install and configure this file for you.

Implementation

AutoTask

In order to implement Google Vision with your NetX assets, you must use NetX's AutoTask system.  You may customize your AutoTask by applying filters which may limit those assets which are auto-tagged, such as by specifying a category or folder, or even configure multiple AutoTask functions with different parameters. 

The example shown below is one implementation which will tag all image assets (e.g. "fileFormatFamily" with a value of "image"). This will apply to those images being imported, and will tag those imported assets within the specified attribute field ("Labels"). This particular AutoTask specifies that Google Vision will look for and tag landmarks or labels up to 10 new attributes, separated by a semicolon:

<matchCriteria type="and">
        <criteria type="action" value="import"></criteria>
        <criteria name="fileFormatFamily" type="attribute" value="image"></criteria>
 </matchCriteria>
<customJob attrName="Labels" className="com.netxposure.products.imageportal.autotask2.impl.GoogleVisionExtractImageJob" delimiter="semi" 
    faces="false" labels="true" landmarks="true" logos="false" text="false" maxLabels="10">
</customJob>

NOTE

In order for Google Vision and its configured AutoTask to take effect, restarting NetX is required. Additionally, if you wish to retroactively apply Google Vision tags to existing attributes, resyncing or reimporting may be required, depending on your AutoTask.

AutoTask Parameters

The example above can be customized using the following parameters: 

NOTE

If an Autotask does not include a faces, labels, landmarks, logos, or text parameter set to "true", no tagging will take place.


NameTypeDefaultDescription
maxLabels

integer

5

Maximum number of labels Google will return. This maximum applies to each Google Vision function (not the aggregate total).

attrName
StringnoneName of a pre-existing Attribute; its type should be text or textArea.
delimiter
StringspaceThis will determine the delimeter used for your list of labels; accepted values are comma, semi (for "semicolon"), or space.
faces
BooleanfalseWhether or not facial expressions will be part of Google Vision's analysis.
labels
BooleanfalseWhether or not labels will be part of Google Vision's analysis.
landmarks
BooleanfalseWhether or not landmarks will be part of Google Vision's analysis.
logos
BooleanfalseWhether or not logos will be part of Google Vision's analysis.
textBooleanfalseWhether text from your images will be pulled into your attribute data.

Proxies

For your photos to be properly tagged using Google Vision, the asset in question must have a proxy file. This may be, in order of preference, a Zoom, Preview, or Thumbnail. This is due to proxy files being more efficient as well as prevent any formatting issues (all proxy files are formatted as JPGs). 

In Action

Using the AutoTask example above, navigate to the AutoTask tab of the Systems area, create a new task, and paste the content in between the subsequent <task></task> brackets: 

Once you have saved your AutoTask, your instance will require a restart in order for the AutoTask to take effect. After a restart has been performed, you may begin importing new image assets and see the Google Vision attributes being applied to the Attribute field indicated by your AutoTask. However, your Attribute values may not be immediately available in your selected Attribute, and may take up to a couple of minutes to see the values reflected.