Use Confluence As a Content Source

SearchUnify can index the pages and blogs stored in your Confluence instance. This article walks you through the process of setting up Confluence as a content source.

PERMISSIONS

• SearchUnify applies Space and Page permissions.

  • Admins can only crawl the Spaces and Pages for which they have view access.

  • Users can only find documents in the Spaces and Pages for which they have view access.

• The Page permissions take precedence between the Space and Page permissions.

  • If an admin has access to a Space and a user has hidden a few Pages in that particular Space, then the admin can crawl all the pages in that Space except the hidden Pages.

  • Users cannot find the hidden Pages in search results even if they have access to that Space.

• Nested permissions are supported up to eight levels.

• The number of responses received from Confluence over a given period depend upon the Confluence's rate limits. You can find the the latest rate limits on their website.

Establish a Connection

1. Navigate to Content Sources and click Add New Content Sources.

   

3. Find "Confluence" through the search box and click Add.

   

4. Give your content source a name.

   

5. Enter the web address of your Confluence instance followed by wiki/ in Client URL.

   

6. Select an Authentication Method.
   • Basic. Select it to crawl the spaces that the Confluence user, whose username and API token has been entered, is allowed to access. Anybody who is part of an organization can authenticate and crawl data. Every user will need to create their own API token. Check out SearchUnify's doc on Create an API Token in Atlassian (Jira and Confluence) or the Confluence doc on Create an API Token.
   • OAuth. Select it to crawl all the spaces and projects to which the client application has access. The level of access and permissions granted to the client application depends on the admin's role and the permissions assigned to them in Confluence. Only a Confluence admin can create the client application. The instructions are in Create an App in Confluence (from SearchUnify) and in Conflgure an Incoming Link (from Confluence).

   

7. Click Connect.

   

Re-Connect

The Authentication screen is displayed when an existing Content Source is opened for editing, as given below. An admin can edit a Content Source for multiple reasons:

• To reauthenticate

• To fix a crawl error

• To change frequency

• To add or remove an object or a field for crawling

• For multiple other reasons

When you edit a Content Source, there can be any one of two cases as listed below:

If the Content Source authentication is successful; a Connected message is displayed.

Case 1: There are no crawl errors and the Content Source authentication is valid.

If the Content Source authentication fails or is disrupted; a Re-Connect button is displayed.

There is a crawl error or the authentication details have changed. In both cases, the SearchUnify Content Source connection must be authenticated again i.e. re-authenticated. To authenticate a Content Source again, enter the authentication details, and click Re-Connect.

Set Up Crawl Frequency

The first crawl is always manual and is performed after configuring the content source. In Choose A Date, select a date to start crawling; the data created after the selected date will be crawled. For now, keep the frequency to its default value Never and click Set and move to the next section.

Select Types and Fields for Indexing

SearchUnify can index Confluence pages and blogs. You can choose to index them both, or select just one of them. You can further index all blog and page fields, or only a few of them.

1. Click to select content fields.

   

2. Use the dropdown in the Name column to add content fields one at a time.

   

3. OPTIONAL. SearchUnify assigns each field a label, type, and either an isSearchable or isFilterable tag. The values don't require a change, but advanced users can edit them.

   

4. Press Save.

   

5. Repeat the steps 2-5 with the second content type.
6. Navigate to By Place.

   

7. Use the index to find your project and check enable for each one of it.

   

8. Press Save.

   

You have successfully installed Confluence as a content source.

Find and Replace

Those on the Q2 '24 release or a later version will notice a new button next to each object on the Rules screen. It resembles a magnifying glass and is labeled "Find and Replace." You can use this feature to find and replace values in a single field or across all fields. The changes occur in the search index, not in your content source.

Find and Replace proves valuable in various scenarios. A common use case is when a product name is altered. Suppose your product name has changed from "SearchUnify" to "SUnify," and you wish for the search result titles to immediately reflect this change.

1. To make the change, click .

2. Now, choose either "All" or a specific content source field from the "Enter Name" dropdown. When "All" is selected, any value in the "Find" column is replaced with the corresponding value in the "Replace" column across all content source fields. If a particular field is chosen, the old value is replaced with the new value solely within the selected field.

3. Enter the value to be replaced in the Find column and the new value in the Replace column. Both columns accept regular expressions.

4. Click Add. You will see a warning if you are replacing a value in all fields.

5. Save the settings.

6. Run a crawl for the updated values to reflect in search results.

After the First Crawl

Return to the Content Sources screen and click in Actions. The number of indexed documents is updated after the crawl is complete. You can view crawl progress in in Actions. Documentation on crawl progress is in View Crawl Logs.

Once the first crawl is complete, click in Actions open the content source for editing, and set a crawl frequency.

1. In Choose a Date, click to fire up a calendar and select a date. Only the data after the selected date is indexed.

   

2. The following options are available for the Frequency field:

   • When Never is selected, the content source is not crawled until an admin opts for a manual crawl on the Content Sources screen.

   • When Minutes is selected, a new dropdown appears where the admin can choose between three values: 15, 20, and 30. Picking 20 means that the content source crawling starts every 20 minutes.

   • When Hours is selected, a new dropdown is displayed where the admin can choose between eight values between 1, 2, 3, 4, 6, 8, 12, and 24. Picking 8 means that the content source crawling starts every 8 hours.

   • When Daily is selected, a new dropdown is displayed where the admin can pick a value between 0 and 23. If 15 is chosen, then the content source crawling starts at 03 p.m. or 1500 hours every single day.

   • When Day of Week is selected, a new dropdown is displayed where the admin can pick a day of the week. If Tuesday is chosen, then content source crawling starts at 0000 hours on every Tuesday.

   • When Day of Month is selected, a new dropdown appears where the admin can select a value between 1 and 30. If 20 is chosen, then content source crawling starts on the 20th of each month.

     It’s recommended to pick a date in the range 1-28. If 30 is chosen, then the crawler may throw an error in February. The error will be “Chosen date will not work for this month.”

   • When Yearly is selected, the content source crawling starts at midnight on 1 January each year.

   

3. Click Set to save crawl frequency settings. On clicking Set, you are taken to the Rules tab.

   

Data Deletion and SU Index

All the data deleted from Confluence is removed from the SearchUnify index within 24 hours.