Python: Getting Secrets from Google Secret Manager

Frequently we run scripts that need to access secure resources. Sometimes developers will hard-code credentials in the code itself. This is obviously a bad idea and can result in secrets being checked into source control by accident and no one wants to be that developer.

In this article, we will take the following approach to store secrets securly and access them with python:

• Create a secret in Google Secret Manager
• Setup the Google Cloud CLI
• Use Python to Access the Secret

Note: There is a charge for using and accessing Google Secret Manger secrets. The charge is very minimal but should be taken into consideration for heavy loads. Review the pricing structure here.

Let's say you need credentials for an api call. One option would be to add those credentials to Google Cloud Secret Manager.

To setup a secret:

1. Log in to the Google Cloud Console.
2. Search for "secret" in the search bar at the top and select Secret Manager in the dropdown.
3. Press the + CREATE SECRET button to add a new secret.
4. Enter a Name for the secret (keep for later).
5. Enter a Secret Value (see below).
6. Press the CREATE SECRET button at the bottom.

For this example, we are going to store login credentials. We can enter the following JSON for the Secret Value. We don't need to create separate secrets for username and password values.

{
    "username": "[the username]",
    "password": "[the password]"
}

The Goolgle Cloud CLI or gcloud is a utility application for creating and managing resources in Google Cloud. We will be using gcloud to access the secret we created above. You could also use gcloud to create secrets instead of using the console.

You may need to Install the CLI if it not already installed. The commands below will help setup the CLI after installation:

# authenticate
gcloud auth login

# set the project
gcloud config set project [the GCP project id]

# set default credentials
# (this will prompt you for your GCP credentials)
gcloud auth application-default login

Here is the python code to access the secret in Secret Manager:

# imports
from google.cloud import secretmanager
import json

# setup variables
project_id = '[the GCP project id]'
secret_id = '[the name of the secret]'
secret_version = '[the version of the secret]'
full_name = 'projects/{}/secrets/{}/versions/{}'.format(project_id, secret_id, secret_version)

# get the secrets
client = secretmanager.SecretManagerServiceClient()
response = client.access_secret_version(request={'name': full_name})
secret_value = response.payload.data.decode('UTF-8')
secret_map = json.loads(secret_value)

username = secret_map['username']
password = secret_map['password']

# use the credentials in your script.

When this code executes, the SecretManagerServiceClient will use gcloud in the background to access the specified secret. Since we already set up the default credentials above, we won't be prompted for our GCP credentails every time the script runs.

This is one way to keep your secrets safe by storing them in Google Secret Manager. Let us know if you have any questions or improvements in the comments below.