Navigating the Google Suite Directory API

Danny Cosson, 4 Oct 2017


Like many small to medium companies, at Fin we use G Suite (formerly Google Apps) for our corporate email, calendar, Google docs and sheets, etc. That means everyone who works here has a Google account set up on our domain, so it’s actually a pretty solid employee database that we already have set up and are keeping up to date.¹

I recently found myself wanting to access these employee user accounts using the Google API. It was painful enough, and I didn’t find any other thorough setup guides online, that I thought it would be helpful to write something up.

The main source of confusion was that many of the Google developer service API’s are designed for traditional OAuth use. For example to sync your Google calendar to your phone, you go through an OAuth login flow to grant a particular app on your phone access to read and write your own calendar data. For our use of accessing the company-wide directory of users and groups, we need to generate credentials that have the right access without any per-user interactive OAuth login step. This is very different pattern — it’s shoehorned into Google’s existing OAuth API auth model but it’s less thoroughly documented, and in places out of date, in the documentation.

For reference, the main docs page on this setup is here. You will need to make changes in both the G Suite Admin console as well as the Google Developers Console.

Step 1: Set up a delegate user with access to the directory.

Even though the primary authorization is done by granting the “Service Account Key” access to the necessary API scopes (Step 3 below), you have to make directory requests on behalf of a user, and that user also needs access to the directory service. You can use your personal Google user, or to be a little more robust you can create a new user for this purpose such as “googledirectories@yourdomain.com”.

Go to the user’s detail page in the G Suite Admin console, click “Show More” at the bottom, and click on the cell that says “0 Admin roles and privileges”. Click “Manage Roles” and assign the “User Management Admin” role to this user (or “Create new roles” first if you want to create a more locked down role with read-only access or something).

img

The permissions on this delegate user need to be equal or broader in scope than you need the API token permissions to be.

Step 2: Set up the Project and Service Account Key in the Google Developers console.

Go to: https://console.developers.google.com

In the top left, there is a dropdown menu with your currently selected project or org², click the dropdown then click the plus button to create a new project, and attach it to your existing billing account. Select the project you’ve just created, and then there are a few things to set up:

Under IAM & Admin -> IAM, click +ADD at the top and add the email address of the user you created, or the existing user you decided to use, in Step 1. Make it a Service Account Actor.

img

Now go to IAM & Admin -> Service Accounts and choose Create Service Account. You have to check Enable G Suite Domain-wide Delegation, to be able to use this as an API key to use the directory API outside of the individual user OAuth flow.

img

For some reason, the Domain-wide delegation requires you to set up the OAuth Consent Screen for your project. However, if this is a new project dedicated to just using the Directory API’s, this OAuth consent screen will never be shown anywhere as far as I can tell so I don’t know why it’s required. Give it any name like “Fin Directory Demo”, then click Create OAuth Consent Screen and fill out the required fields there, using your own email or any group's email you want. Saving that form takes you back to the Create Service Account form which you can now complete.

img

After you click Create you should be back onto a page at API's & Services -> Credentials where you now have one OAuth credential. However, this is not a real credential.³ So you need to click Create Credentials -> Service Account Key. Choose your new service account and the JSON option, then Create. It will download a file and this is your real credentials file.

Now you’ll see two rows of credentials on the credentials page, a service key and an oauth client id. Copy the Client ID part from the top row (in the screenshot below, mine is 10045…).

img

Before you leave this console, go to API's & Services -> Library and search for Admin SDK. Click Enable at the top to enable usage in this project (the Directory API's live under this broader Admin SDK category).

img

Step 3: Grant the Service Account the scopes it needs.

Now back to the Admin console. Go into Security -> Show More -> Advanced Settings -> Manage API Client Access. Under Client Name, paste the Client ID you copied near the end of Step 2. Then add the Admin directory API scopes you need to use. Again, these have to be a subset of the permissions you set for the delegate user in Step 1. Click Authorize to save.

img

Step 4: Initialize the service in your code

This part is written out pretty clearly in the Google docs, but they reference the old .p12 format (which the console now says is deprecated) instead of the json format. Set up the client with something like this python example:

Here I’ll set the env var GOOGLE_KEYFILE_JSON to be the json contents of the credentials key file you downloaded above. To export it wrap the json object in single quotes and combine the whole thing onto one line, like so:

  
    $ export GOOGLE_KEYFILE_JSON='{ "type": "service_account",
    "project_id": "fin-directory-demo", "private_key_id": ... }'
  

The delegate_user_email is the delegate user from Step 1. Now you're ready to use the service:

  
service = 
authenticated_directory_service('googledirectories@yourdomain.com') user = 
service.users().get(userKey='anyuser@yourdomain.com').execute()
  

That returns a json object representing the user. See the documentation for all the API calls available.

And that’s it!


¹ There are of course many other dedicated directory tools we could have be using instead. Microsoft ActiveDirectory and many types of LDAP servers are both very common but these are traditionally self hosted which obviously introduces a new operational burden. AWS offers an ActiveDirectory cloud service but don’t seem to expect anyone to use it beyond mirroring another ActiveDirectory to it, and it doesn’t even integrate well into their IAM users & roles system. So file that away in the bucket of weird corporate AWS services that solve problems I may never understand.

The web-native options like Okta and OneLogin are probably the best bet for a startup like us, they provide their own directory and also have built integrations to enable single sign on to a variety of other web services. But the recent OneLogin breach was pretty scary, plus Google already supports their own Oauth-based as well as SAML SSO, so we haven’t felt enough pain to make the jump to another dedicated tool.

² It took me a while to figure out the navigation hierarchy of these menus. Clicking the hamburger menu icon in the top left lets you switch between “API & Services” and “IAM & Admin” sections, both of which you’ll need to use here. The top left drop-down can either be an organization or a project. When it’s an organization most of the sub menu items are grayed out which confused me as well.

³ If you click the “download json” link for this oauth client id, you’ll get a file named client_secrent_<client_id>.json. But if you open it there is no secret to be found. ¯\_(ツ)_/¯