Overview

This document aims at explaining the Ephesoft Encryption framework. This framework enables Ephesoft to encrypt/decrypt the data persisted in XML files and index files while batch processing. In order to achieve encryption/decryption capabilities Ephesoft maintain encryption/decryption keys at following levels:

1. Application Level Key
2. Batch Class Level Key
3. Batch Instance Level Key
4. Fuzzy Index
5. Lucene Index Key
6. Test KV Key
7. Test Classification Key
8. Test-Advance KV Key
9. Test-Table Key

Purpose

Ephesoft Encryption framework enables Ephesoft to secure data from any unauthorized access.

Key Features

1. Every Application is identified by a unique Key.
2. All the data persisted in HOCR.xml, Batch.xml and index files etc. is stored on disk in encrypted form to avoid any unauthorized access of critical data while processing.
3. All the learned samples and learned index files are re-generated whenever there is a change in encryption algorithm.
4. Any change in Batch Class Key without changing the encryption algorithm does not affect the learned files and thus they are not regenerated in such case.
5. Batch Class can be imported using the existing Keys as well as by generating the new key from UI.
6. Only authorized user will be able to decrypt the encrypted Batch.xml file using web service API. An authorized user is one who belongs to the groups/roles assigned to the batch class on which batches are processed.
7. No Batch can be decrypted on an application other than the application on which it was processed.
8. Only a super admin can change the encryption key of a batch class.

Hashing Algorithm

Keys are generated using Hashing algorithm on the data available for generating Keys.

””Key Length	Hashing Algorithm
128	MD5
256	SHA-256

Encryption Algorithm

AES algorithm is backbone of Ephesoft Encryption framework and it is used to encrypt and decrypt the data. Super admin has flexibility to choose between AES-128 / AES-256 bit algorithm.

Application Level Key- The Crux of Encryption

Application level key concept is very similar to a digital signature or a digital certificate which uniquely identifies an application/entity/organization/individual. Application level key in Ephesoft identifies an application belonging to a customer uniquely.

The Key (identity) provided for application key is used as a secure password for all the key stores generated through-out the process i.e. application level key is responsible for validating that the keys/key-stores being used throughout application at different stages belongs to a particular application (key/key-stores belonging to a particular application will not work on any other unauthorized application).

This key will also be responsible for authentication during the Batch XML decryption, which will not allow the Batch XML to decrypt outside the authorized application (unauthorized applications will not be able to decrypt Batch XML belonging to some other application).

Levels of Secret Keys

Level	Location	Mechanism	Key Length
Application	<Shared-Folders>/ephesoft-key-store-file/ephesoft.keystore	Can be generated only once from UI.	128
Batch Class	<Batch-Class-folder>/ephesoft-key-store-file/ephesoft.keystore	Configurable from UI.	128/256
Batch Instance	<ephesoft-system-folder>/<Batch-Instance-ID-Folder>/ephesoft-key-store-file/ephesoft.keystore	Generated dynamically using Application Level Key and Batch Class Key.	Length of Batch Class Key.
Fuzzy-DB Index	<Batch-Class-Folder>/<fuzzy-index-folder>/ephesoft-key-store-file/ephesoft.keystore	Generated dynamically using Application Level Key and Batch Class Key.	Length of Batch Class Key.
Lucene Key	<Batch-Class-Folder>/<lucene-search-classification-sample>/ephesoft-key-store-file/ephesoft.keystore	Generated dynamically using Application Level Key and Batch Class Key.	Length of Batch Class Key.
Test KV	<Batch-Class-Folder>/<test-KV-Folder >/ephesoft-key-store-file/ephesoft.keystore	Generated dynamically using Application Level Key and Batch Class Key.	Length of Batch Class Key.
Test Advance KV	<Batch-Class-Folder>/<test-Advance-KV-Folder>/ephesoft-key-store-file/ephesoft.keystore	Generated dynamically using Application Level Key and Batch Class Key.	Length of Batch Class Key.
Test Classification	<Batch-Class-Folder>/<test-Content-Classification-Folder>/ephesoft-key-store-file/ephesoft.keystore	Generated dynamically using Application Level Key and Batch Class Key.	Length of Batch Class Key.
Test Table	<Batch-Class-Folder>/<test-table >/ephesoft-key-store-file/ephesoft.keystore	Generated dynamically using Application Level Key and Batch Class Key.	Length of Batch Class Key.

Assumptions

1. Application Key should be generated before applying the encryption algorithm on a batch class.
2. The user needs to make a change in the Scripting plugin for the plugin execution.

UI Configurations

Application Key can be generated from the System Config tab by the Super Admin.

Batch Class Key can be generated by the Super Admin of the Application from the Batch Class Management Screen.

Encryption can be enabled/disabled during Batch Class Import. In addition to this imported Batch Class can also use an existing key present inside the exported batch class.

If user wants to enable encryption during batch class import or wants to change key of existing encrypted batch class then while importing re-learning all the files, key/key-store generation will take place which may take some time.

The use exiting encryption keys can be used when a user wants to use already generated keys/key-stores in batch class because user does not want to generate new keys such that no re-learning, key/key-stores generation is done. But this feature works only if the application level key i.e. application’s identity of batch class being imported is same as application to which batch class is being imported. This validation is done so that no un-authorized application can import a batch class which does not belong to same organization. e.g.

A batch class BCx belongs to organization A and another organization B is trying to import the batch class BCx using existing keys into their application. Since organization A is different from organization B thus organization B must not be able to use the Batch class which has already been encrypted by another organization. This is validated using Applications identity i.e. application key since application key being used by an organization/application is unique and each application key denotes an organization/application uniquely.

In case if one wants to import/export an encrypted batch class between different environments like Production or Development for same organization then application identity on both Production and Development environment must be same. This will validate that Batch Class belongs to same organization.

To achieve this one will have to use same Application key on all the environments in organization as this key is an identity key and identifies if application belongs to same organization.

If user wants to disable encryption during batch class import of existing encrypted batch class then while importing re-learning all the files, key/key-store generation will take place which may take some time.

Error Messages

Error Messages	Probable Cause
‘Key store file doesn’t exist.	Any of key required for encryption/decryption doesn’t exist.
Keys required for generating the dynamic key doesn’t exist	Any of the application key/ Batch Class Key required for generating the dynamic key doesn’t exist.

Backup and Recovery

1. To recover the data in case application key is lost. It is recommended that user should store the application key file in a secure storage device.
2. It is also recommended to secure the Key-Store file unique password (key_password) from encryption_key_metdata table from the DB.

Ephesoft Encryption Framework Web Service API

Overview

This document aims at explaining the Ephesoft Encryption Framework Web Service API. It talks about primarily two web services namely:

DecryptBatchXml

This web-service helps to decrypt the batch xml file present in the final drop folder. The exported batch xml to be decrypted contains a key which is required for decrypting the batch xml in a Signature tag. This batch XML can only be decrypted on the client’s application on which the batch was executed; this is because of the uniqueness in the Signature which is determined by Ephesoft application in use

Web Service URL

[ http://{serverName}:{port}/dcma/rest/importBatchClass]

Sample client code using apache commons http client:-

private static void decryptBatchXml() {

HttpClient client = new HttpClient();

String url = “http://localhost:8080/dcma/rest/decryptBatchXml”;

PostMethod mPost = new PostMethod(url);

mPost.setDoAuthentication(true);

// Input zip file for importing batch class.

File file1 = new File(“C:\\sample\\BI1_batch.xml”);

Part[] parts = new Part[1];

try {

parts[0] = new FilePart(file1.getName(), file1);

MultipartRequestEntity entity = new MultipartRequestEntity(parts, mPost.getParams());

mPost.setRequestEntity(entity);

int statusCode = client.executeMethod(mPost);

if (statusCode == 200) {

System.out.println(“Batch XML decrypted successfully”);

} else if (statusCode == 403) {

System.out.println(“Invalid username/password.”);

} else {

System.out.println(mPost.getResponseBodyAsString());

}

} catch (FileNotFoundException e) {

System.out.println(“File not found for processing.”);

} catch (HttpException e) {

e.printStackTrace();

} catch (IOException e) {

e.printStackTrace();

} finally {

if (mPost != null) {

mPost.releaseConnection();

}

}

}

ImportBatchClass

This API is used for importing batch class to the Ephesoft. This API takes XML as input parameters and exported batch class zip file as an input. According to the data present in the input XML file and the zip of batch xml a new batch class is imported into Ephesoft. If there is any encryption applied on the batch class to be exported then while importing the batch class Ephesoft can either retain its encryption or set any new encryption as specified in the input XML file. Encryption from a batch class can even be removed while importing it.

Web Service URL

[ http://{serverName}:{port}/dcma/rest/importBatchClass]

Input Parameter	Values	Descriptions
RolesImported	Either “true”/”false”	This value is used for importing roles with batch class or not.
EmailAccounts	Either “true”/”false”	This value is used for importing email accounts with batch class or not.
UseSource	Either “true”/”false”	This value is used for saving the information of source batch class to be imported
Name	This value should not be empty	This value is used to configure the batch class name of the imported batch class.
Description	This value should not be empty	This value is used to configure the description of the imported batch class.
Priority	This value should lie in between 1 to 100.	This value indicates the priority of batch class.
UseExisting	Either “true”/”false”	This value is used for overwrite the existing batch class with new batch class.
UncFolder	This value should not be empty and have any string value that specified directory path	These values specify the UNC folder path for batch class to be imported along with batch class.
Script	This tag is configured for ScriptFile to be imported	This tag is configured for which Script file to be imported
Folder	This tag is configured for Folder to be imported	This tag is configured for which folder to be imported along with batch class
UseKey	Either “true”/”false”	This tag decides if the encryption of exported batch class needs to be retained or not. If “true” then the encryption in exported batch class is retained (if no encryption was there in exported batch class then same will be in imported batch class). If “false” then other settings will be used for generating key.
BatchClassKey	String value to be used as batch class key.	The value for batch class key. If algorithm is “NONE” then no value for it is required.
EncryptionAlgorithm	Name of encryption algorithm: NONE AES_128 AES_256	The type of algorithm to be used for encryption. If “NONE” is set for encryption algorithm then all encryption from batch class is removed. If UseKey value is set to true then this tag won’t change anything.

Checklist:-

1. If UseExisting is “true”, existing batch class will be overwritten.
2. If UseExisting is “false”, new batch class will be created.
3. If UseSource is “true”, new batch class will have same Name, Description and Priority as source batch class.
4. If UseSource is “false”, new batch class will have property like Name, Description and Priority as configured in input xml.

SampleInputXML:

<ImportBatchClassOptions>

<RolesImported>false</RolesImported>

<EmailAccounts>true</EmailAccounts>

<UseSource>false</UseSource>

<Name>BatchClassName</Name>

<Description>Description</Description>

<Priority>10</Priority>

<UseExisting>true</UseExisting>

<UncFolder>C:\ephesoft-data\Test-UNC</UncFolder>

<UseKey>false</UseKey>

<BatchClassKey>key</BatchClassKey>

<EncryptionAlgorithm>NONE</EncryptionAlgorithm>

<BatchClassDefinition>

<Scripts>

<Script>

<FileName>ScriptDocumentAssembler.java</FileName>

<Selected>true</Selected>

</Script>

<Script>

<FileName>ScriptPageProcessing.java</FileName>

<Selected>true</Selected>

</Script>

</Scripts>

<Folders>

<Folder>

<FileName>image-classification-sample</FileName>

<Selected>false</Selected>

</Folder>

</Folders>

<BatchClassModules>

<BatchClassModule>

<ModuleName></ModuleName>

<PluginConfiguration>true</PluginConfiguration>

</BatchClassModule>

</BatchClassModules>

</BatchClassDefinition>

</ImportBatchClassOptions>

Sample client code using apache commons http client:-

private static void importBatchClass() {

HttpClient client = new HttpClient();

String url = “http://localhost:8080/dcma/rest/importBatchClass“;

PostMethod mPost = new PostMethod(url);

mPost.setDoAuthentication(true);

// Input XML for adding parameter.

File file1 = new File(“C:\\sample\\importbatchclass.xml”);

// Input zip file for importing batch class.

File file2 = new File(“C:\\sample\\BC1_050712_1714.zip”);

Part[] parts = new Part[2];

try {

parts[0] = new FilePart(file1.getName(), file1);

parts[1] = new FilePart(file2.getName(), file2);

MultipartRequestEntity entity = new MultipartRequestEntity(parts, mPost.getParams());

mPost.setRequestEntity(entity);

int statusCode = client.executeMethod(mPost);

if (statusCode == 200) {

System.out.println(“Batch class imported successfully”);

} else if (statusCode == 403) {

System.out.println(“Invalid username/password.”);

} else {

System.out.println(mPost.getResponseBodyAsString());

}

} catch (FileNotFoundException e) {

System.out.println(“File not found for processing.”);

} catch (HttpException e) {

e.printStackTrace();

} catch (IOException e) {

e.printStackTrace();

} finally {

if (mPost != null) {

mPost.releaseConnection();

}

}

}

Ephesoft License Server Failover Mechanism

Overview

Ephesoft failover mechanism for license servers provides high-availability in case any of the license server crashes. In current multi-server environment, license server runs on single machine and other client machines communicate with that server for their license validation/verification. This model works fine until license server is up and running. As soon as the hosted license server fails, all clients encounter license validation/verification problems. In addition to this all the activities that require license validation/verification come to a halt. With this new enhancement, Ephesoft application will have an always available environment where even if one license server fails then one of the backup license servers will immediately come into play and will start fulfilling all the license client requests. This transition of license server will happen on the fly (dynamically) and so smoothly that a user will never know that a license server has failed as one of backup license server will immediately start processing the license validation/verification requests. It will prevent batches from going into ERROR state and will avoid task failures like web service requests, login requests etc. Thus user experiences no disruption in services. There is no down time due to swapping between major minor license servers.

Configuration

Steps for configuring the feature

To configure Failover mechanism in multi-server environment, a user will need to install fresh failover license with equal potential on minimum of two servers available in multi-server environment. This feature doesnâ€t work with older licenses.

Working

To start using this feature, user will need to install failover license on minimum of two servers. After installing the license restart all the servers one by one. Once the servers are up user can verify whether failover mechanism is activated or not with the help of License Details tab available under System Config tab. On license detail page one more property License Server Failover Switch will be added to show failover mechanism status.

Failover Mechanism=ON

Once all the servers have been successfully started in failover mode, all license requests will be processed from single license server (server A) as depicted in below diagram.

Once the server (server A) goes down Ephesoft failover mechanism will immediately start processing requests from another available active license server in multi-server environment. This will happen on the fly and without any disruption of normal batch execution or of any other service that require license verification/validation. Scenario when server A fails is shown in below diagram.

In this way a user need not to worry about batches going into error state when one of license server goes offline, as Ephesoft License server failover mechanism will automatically handle the situation by judiciously using potential of multiple license severs available in multi-server environment. This feature is very useful when a user needs to stop a license server for maintenance purpose as when one license server goes offline all client depending on license server will continue to work by switching to another available active license servers.

NOTE: If older licenses are installed with failover mechanism feature. Failover mechanism wonâ€t work as license verification will be done in same way as it happened earlier.

Following diagram demonstrates situation when License Failover mechanism is not in place:

Client A is dependent on server A and client B and client C are dependent on server B for licensing needs as specified in their license-client.properties file.

Dependency

To get this feature working properly in multi-server environment, heart beat module should be running on all the servers and all the servers need to be interconnected with high availability network.

Troubleshooting

Following are few common error messages seen in log file due to improper configuration and execution:

S no.	Error message	Possible root cause
1	CPU limit exhausted. Can’t allocate more CPUs.	The total count of CPU core available in all machine in multi-server environment is greater than count specified in license installed. Get a new license with correct CPU core amount.
2	Invalid license for multi-server environment setup.	Either mac id of server was not included or it was entered wrong while generating license. Please verify mac id while generating license.