Use YouTube As a Content Source

Index YouTube videos and make them searchable for your employees and customers. SearchUnify can index the videos uploaded on your channel and the channels that you have subscribed to.

PERMISSIONS

  • You have to have a YouTube account.

  • To index subtitles and comments on your YouTube videos, allow SearchUnify to "see, edit, and permanently delete your YouTube videos, ratings, comments, and captions" during authentication.

  • With Google OAuth 2.0-based authentication in new SearchUnify instances, an API rate limit of 10,000 grants per day is applicable. It might impact the crawling in SearchUnify. To manage it, try turning captions off to crawl up to 4800 videos.

Establish a Connection

  1. Navigate to Content Sources.

  2. Click Add New Content Source.

  1. From the search box, find YouTube and click Add.

  2. Under the Authentication tab, enter details. If you are on the first quarterly release of 2024 or a later release, then enter the name of the content source, pick a content language, and enter the Cliend ID and Client Secret of the OAuth 2.0 app.

  3. Administrators on a release older than Q1.24 can enter the name of the content source, pick a content language, and click Connect. A permissions window pops up asking for required permissions. Click Allow.

  4. If you are already logged into YouTube (or Google), you will be prompted to give SearchUnify to access "see, edit, and permanently delete your YouTube videos, ratings, comments, and captions" and "view your YouTube account". The first access is essential to index subtitles and comments. Click .

  5. A "Connection Succeeded" message greets you. Click Next.

Google hasn't verified this app

If you encounter an error where Google indicates that the app has not been verified, please disregard the warning and continue with the authentication process. This issue is slated for resolution in future updates.

Set Up Crawl Frequency

The first crawl is always manual and is performed after configuring the content source. In Choose A Date, select a date to start crawling; the data created after the selected date will be crawled. For now, keep the frequency to its default value Never and click Set and move to the next section.

Select Fields and Channels for Indexing

YouTube videos have fields, such as titles, descriptions, and channel names. You can index them all or a selection of them. This section shows how to find fields for indexing in By Content Type. After that, the process to pick channels for indexing through By Channels is described.

Related

How Field Mapping Works Between YouTube and SearchUnify

  1. To start selection, click .

  2. You will discover that all available fields have been selected. The safest path forward is to click Apply. However, if you are an advanced user or a developer, you can remove fields, such as channel_id and comment, and change the Label and Type of a field. Neither change is recommended for most users.

  3. Clicking Apply brings you back to By Channels where next to is .
  4. is used to find and replace values in a field. Think of it this way. Your flagship product has recently been named from "SearchUnify" to "s.e.a.r.c.h.u.n.i.f.y". You want the search result titles to display s.e.a.r.c.h.u.n.i.f.y immediately. to the rescue. Click it, find title in Name, write the regular expression for the old product name in regex, and the regular expression for its replacement in replace. Click Add and then Save. Each instance of "SearchUnify" in title on the search results page will be swapped with "s.e.a.r.c.h.u.n.i.f.y". It's important to mention here that makes changes in the SearchUnify index, not YouTube. On clicking the end-users will continue to encounter good old "SearchUnify."
  5. Navigate to By Channels and use the index to find channels. CCTV is listed in C and SearchUnify in S. The search function can be used to find a channel from a long list.

  6. Check Enable and click Save.

Crawl Playlists Instead of Channels

An alternative method to work with YouTube is to index playlists instead of channels. By Playlists can replace By Channels if you write to support@searchunify.com. By Playlists gives you more flexibility in crawling because you can limit search to the videos inside selected playlists. When By Playlists is active the By Channels tab disappears.

After the First Crawl

Return to the Content Sources screen and click in Actions. The number of indexed documents is updated after the crawl is complete. You can view crawl progress in in Actions. Documentation on crawl progress is in View Crawl Logs.

NOTE 1

Review the settings in Rules if there is no progress in Crawl Logs.

NOTE 2

For Mamba '22 and newer instances, search isn't impacted during a crawl. However, in older instances, some documents remain inaccessible while a crawl is going on.

Once the first crawl is complete, click in Actions open the content source for editing, and set a crawl frequency.

  1. In Choose a Date, click to fire up a calendar and select a date. Only the data after the selected date is indexed.

  2. Use the Frequency dropdown to select how often SearchUnify should index the data. For illustration, the frequency has been set to Weekly and Tuesday has been chosen as the crawling day. Whenever the Frequency is other than Never, a third dropdown appears where you can specify the interval. Also, whenever Frequency is set to Hourly, then manual crawls are disabled.

  3. Click Set to save crawl frequency settings. On clicking Set, you are taken to the Rules tab.

Known Issue

In Frequency Crawl, only the new videos are crawled. Any changes in the previously-crawled videos do not reflect in search results.

OAuth 2.0 Setup

If you use Q1.24 or a newer version of SearchUnify, then your YouTube and Google Drive content sources will stop working if you haven't set up OAuth 2.0 and you'll see an error: 

Copy

Error

You need to set up your Google OAuth to continue using this content source. Click here to know more.

You will see the following errors on your YouTube and Google Drive content sources in case you haven't authenticated them with OAuth 2.0.

Set up OAuth 2.0 on your google account and re-authenticate your content sources using the client ID and client secret.

Help Article: Setting up OAuth 2.0

Last updated: Thursday, April 25, 2024

Or, send us your review at help-feedback@searchunify.com