Handling Enhanced Data Stores

TheNews → Data Sources → Enhanced Data Storespage lists all Enhanced Data Stores in your organization. From here you can create new stores, browse their contents, manage files, and delete stores.

SeeEnhanced Data Storesfor a conceptual overview of what they are and how they work.

Creating an Enhanced Data Store

1. Go toNews → Data Sources → Enhanced Data Stores.
2. ClickCreate.
3. Work through the setup wizard.

Step 1 — Choose a Content Source

Select where documents will come from:

• User Files— You upload files manually. The store is empty to start with; you add files after creation.
• Latest News from a Data Source— The store fetches the most recent articles from an existing online data source. You select the source and an optional cutoff date.
• Search Results from a Data Source— The store runs a keyword query against an existing data source and indexes the results. You provide the search term and select the source.

For the last two options, proceed to Step 2 to select the data source.

Step 2 — Select the Data Source(online sources only)

Pick an existing online data source from your organization. Only sources that support the required capability for the chosen content type (Latest or Search) are shown.

Step 3 — Choose a Transform

Select how documents should be processed before indexing. TheDefaulttransform is the right choice for most scenarios — it auto-detects the format and extracts text. For specialized content types (audio, video, scanned PDFs), choose the appropriate transform.

SeeEnhanced Data Stores → Transformsfor descriptions of all available transforms.

Step 4 — Choose a Splitter

Select how documents are divided into chunks.Paragraph stackingis the default and works well for most documents. ChooseWhole documentfor very short files, orHierarchicalfor long structured documents with clear headings.

SeeEnhanced Data Stores → Splittersfor descriptions of all splitter options.

Step 5 — Choose an Index Technology and Embedding Model

SelectFlash(recommended) as the index technology. Then pick an embedding model from the list. The embedding model determines how text is converted into semantic vectors; choose a model that supports the primary language of your content.

Step 6 — Name the Store

Enter a name and clickCreate. The store is created and appears in the list. If you choseUser Files, you can now open the store and start uploading.

Uploading Files

1. Open an Enhanced Data Store of typeUser Filesfrom theNews → Data Sources → Enhanced Data Storeslist.
2. ClickUpload(orFilesto go to the file management view).
3. ClickUploadon the files page, or drag and drop files onto the page.
4. Select one or more files from your computer.

Files are uploaded immediately. Each file then enters a processing queue and progresses through these statuses:

The page refreshes automatically every 10 seconds while files are being processed, so you can monitor progress in real time.

Searching Within an Enhanced Data Store

Once files have theReadystatus, you can search within the store:

• From theEnhanced Data Storeslist page, clickSearchnext to the store.
• From the files view of a User Files store, clickSearchin the toolbar.

Both links take you toNews → Searchwith the Enhanced Data Store pre-selected as the source.

Reindexing Files

Reindexing re-processes a file with a (possibly different) transform and splitter and rebuilds its index entry. Use reindexing when:

• You want to apply a different transform (for example, switching to audio transcription after uploading a podcast).
• A file failed processing and you want to retry.
• You have updated the splitter settings and want to rebuild all chunks.

Reindex a single file

1. Open the files view of the Enhanced Data Store.
2. Click theReindexbutton (sync icon) next to the file.
3. In the dialog, optionally change the transform and/or splitter.
4. ClickReindex.

Reindex multiple files

1. Select the files you want to reindex using the checkboxes.
2. ClickReindex (N)in the selection toolbar.
3. In the dialog, optionally change the transform and/or splitter.
4. ClickReindex.

After reindexing starts, the affected files revert toWaitingstatus and are processed again.

Downloading Files

To download a file from a User Files store:

1. Open the files view.
2. Click theDownloadbutton (down-arrow icon) next to the file.

The file is downloaded to your computer using its original file name.

Deleting Files

Delete a single file

1. Open the files view.
2. Click theDeletebutton (bin icon) next to the file.
3. Confirm the deletion.

Delete multiple files

1. Select the files you want to remove using the checkboxes.
2. UseSelect allin the toolbar to select every file in the current view if needed.
3. ClickDelete (N)in the selection toolbar.
4. Confirm the deletion.

Deleted files are removed from both the file store and the search index. This action is permanent.

Renaming an Enhanced Data Store

1. Go toNews → Data Sources → Enhanced Data Stores.
2. ClickRenamenext to the store.
3. Enter the new name and clickSave.

Deleting an Enhanced Data Store

Deleting a store removes it along with all its files and index data.

1. Open the files view of the store you want to delete.
2. ClickDeletein the toolbar.
3. Confirm the deletion.

You can also delete a store from inside its files view. Deletion navigates you back to the Enhanced Data Stores list.

Warning:Deleting an Enhanced Data Store is permanent. All uploaded files and indexed content are removed and cannot be recovered.