SearchAssist enables content ingestion from various sources like web pages, files, and third-party applications using crawlers and connectors. However, for applications without a pre-built connector in SearchAssist, the Custom Content Connector can be used. This connector leverages the application’s REST APIs for indexing content, facilitated through the Custom Connector Service provided by SearchAssist.

Overview

The custom connector service acts as a middleware between SearchAssist and the third-party application. It retrieves content from the application and converts it into an indexable format for SearchAssist. To fetch the content, the service must be configured with the necessary details, such as REST API endpoints, authorization details, etc. A reference implementation of this service is available for download, providing a base for integrating with third-party applications.

Custom Connector Service Setup

The custom connector service implementation is a basic implementation and serves as a template for integration.  It can be enhanced as per your requirements. The current implementation of the service only supports basic authentication. Follow the steps listed below to get started.

1. Download the Service:
   1. Obtain the skeleton custom connector service from the provided link.
2. Service Configuration:
   1. Modify the config.json file to include the configuration details of the application and the content fields. 
   2. To enable communication between SearchAssist and the service, a basic auth mechanism is used. In the current implementation, the password must be set in the .env file in the root folder of the service. You can rename the existing .env.sample file available in the root folder to the .env file and add your Auth value to the Authorization key. 
3. Service Implementation Changes:
   1. Make changes to the service implementation, if required. Refer to the readme file available with the service for more details. 
4. Host the Service:
   1. After configuring and making necessary changes to the service, host the service on your preferred platform.

Steps to Enable the Custom Content Connector on SearchAssist

1. Service Setup

   • Modify the service to handle authorization and content indexing from the third-party application and host it.

2. Configure Custom Connector in SearchAssist

   • Navigate to the Connectors section under the Sources page.
   • Select Custom Connector and configure it with the following details:
     • Name: A unique name for the connector.
     • Endpoint: The URL of the hosted service. This is the endpoint of the API that provides the content. Also, specify the HTTP method for the API. For the default implementation of the Connector Service, the endpoint is
       GET http://<serverip>/getContent
     • Under the Headers, add the following key and value. The value is the auth value set in the .env file in the service. This is required to establish secure communication with the service. For the default implementation of the service, use the following Key as it is. If any changes are made to the auth mechanism of the service, change the Headers accordingly.
       Key: Authorization
       Value: <Your-Auth-Key>
     • Click on Proceed to complete the setup. 

Content Synchronization

To synchronize content, follow these steps:

1. Navigate to the Overview tab of the custom connector configuration page.
2. Click Synchronize Content Now.
   • The service retrieves batches of 30 documents from the third-party application and sends them to SearchAssist.
   • SearchAssist processes each batch, indexing the content into the Search Index and Answer Index. Synchronization is complete when all batches are processed. 

 Access Control

SearchAssist also enables RACL on the content ingested via the custom connector. SearchAssist learns about the permissions-related field in the custom connector using the sys_racl field in the config file in the Connector Service. The data fetched from the field specified as sys_racl field in the config file is saved as the permission entity. To add any user to the permission entity, use the Permission Entity APIs. 

For instance, if the access information is in the permissions field for your content, set the sys_racl field as shown below in the Connector Service Config.

A permission entity is created for every value fetched from the permissions field in the content. Use the Permission Entity APIs and add the users to the permission entity. Only users added to a given permission entity will have access to the content specific to that permission entity.