Best Practices for Storing Keys and Credentials for 3rd Party Systems in Drupal

Submitted by Jaypan on

Note: This blog was written without the use of AI, other than a proofread for grammar and spelling mistakes.

Drupal, being an excellent framework for integrating various unrelated systems, often requires usage of 3rd party keys. For example, when integrating with Google Wallet API, a developer creates an authorization key on Google, downloads it, then sets up their system to use this key to generate a JWT token that is passed back to Google. This authorization key needs to be made available to the system when building packages, yet also needs to be stored securely so that it does not fall into nefarious hands. 

The Key Module

The key module is made to handle this issue. The key module allows for creating a "key" that is stored in config with an API for retrieval that can be used elsewhere in Drupal code. In Drupal, active configuration is stored in the database. Configuration can be migrated between environments by exporting the configuration from the source database to file, migrating the files to the target environment, then importing the migrated files into the target environment's active configuration in the database.

The migration is generally handled with Git; exported config files are tracked in Git, with Git commits being pushed between the environments. This does open up a security issue; if a password/authorization is stored in Git, it leaves it exposed to anyone with access to this repo. 

Fortunately, the key module is set up to handle this situation. With this module, when creating a key, the user is provided with an option for provider type; the provider being the method by which the key will be stored. The default option is "Configuration", which means the key is stored in the database configuration, and will be exported to file, and therefore end up in Git. This is not what we want.

There are three other options however, each of which will store the credentials outside of configuration:

  • State: Drupal core provides the State API. This API stores values in the database, but entirely separately from configuration. This means that the data will not be part of configuration exports, however, it will be part of database exports. As databases are often exported and provided to developers, this method, while slightly preferable to storing keys in configuration, still leaves open a larger potential for accidentally exposing the data to an unintended party, and therefore isn't recommended.
  • Environment: This stores the value in an environment variable, which is a means of storing a value in the operating system's memory. Environment variables are often injected in CI/CD pipelines, however the values are then available to may be accessible to other processes in the same server space, which can expose the keys to other compromised software.
  • File: With this provider, the authorization credential is stored in a file on the filesystem1. This provider method is the most secure, as the credentials are not stored in the database, nor in configuration. However, the ../keys directory, and specifically the keys file(s), must be added to .gitignore, so that these files never end up in Git2.

For secure keys, I always use the file provider, as this is the most secure. It does however require that the key file be manually uploaded to each server, in the same location on the server relative to the webroot.

Keys for Production and Sandbox Environments

Often, 3rd party APIs will provide a sandbox environment that can be used during development for testing, so developers can configure the system to work before pushing it live. In this case, the keys used to interact with the sandbox and production environments will differ. This introduces a requirement for key management and end point management on different environments. This creates the following requirements:

  • Development environments must point at the endpoint https://sandbox.example.com/endpoint.
  • The production environment must point at the endpoint https://www.example.com/endpoint. 
  • Development environments must use a key file downloaded for the 3rd party sandbox API
  • The production environment must use a key file downloaded for the 3rd party production API

Achieving this goal requires setting up a key file on both development and production servers, creating a Drupal key using the key module, and setting up a configuration split for the production environment to allow for differing endpoints. This is done as follows:

  1. In your local environment, create a directory outside the webroot for keys, and save the key file retrieved from the 3rd party API sandbox server to this directory.
  2. Create a new key in Drupal using the key module, setting the provider type as "file". Set the filepath as the path to the file saved in step 1.
  3. On the production server, create a directory outside your webroot, mirroring the directory you created in step one locally. Save the key file retrieved from the 3rd party API production server to the same location, and with the same filename as the file in step one. This means that relative to the webroot in both environments, the key file for the respective environment will have the same name, meaning the key module will get the value for the current environment as necessary.
  4. In your local installation, configure it with the staging endpoint. How this is done will depend on how you are integrating with that 3rd party API, and whatever Drupal module you are using to integrate with that API.
  5. Install the Configuration Split module.
  6. Set up a configuration split for the production environment.
  7. Export your configuration.
  8. Do a text search of the exported configuration files for the sandbox endpoint. It will be found in [configuration key].yml. Make note of the configuration key for the next step. It's the full filename, minus the .yml extension.
  9. In the config split for production, under "partial split", select the configuration key from step 8, and save.
  10. Export your configuration to get this new configuration.
  11. Commit your configuration to Git.
  12. Migrate your configuration to the production server.
  13. Import the configuration to your production server.
  14. Configure your production environment endpoint on the production server to point at the 3rd party API production endpoint, in the same way you configured it on the local development server. This means that the production server now will communicate with the production endpoint, using the production key from step 3.
  15. Export the production server configuration to file on the production server, to capture the updated endpoint configuration. This should create a new file in the directory you set for production configuration when creating the production config split in step 6.
  16. Commit the newly exported file to Git, and push back to your central repo.

Note that this method will require that the keys directory be set up for each development environment before the sandbox can be used for that environment. In the circumstance that other environments have different API endpoints to communicate with, additional config splits can be set up in the same manner as the production config split described above.

Integrating with the Key Module in Other Modules

Often as a module developer, it's beneficial to integrate with the Key module. For example, I have recently released the PK Pass Integration API module that integrates Drupal with Apple Wallet. (I'm currently working on a Google Wallet equivalent). This module requires a .p12 file, downloaded from Apple, to create the wallet item package downloaded by the user's device. 

In the configuration form for the module, I set up the following form element:

$form['apple_cert_key'] = [
  // Creates a select element listing keys created with the Key module.
  '#type' => 'key_select',
  '#title' => $this->t('Apple Pass Certificate Key'),
  '#description' => $this->t('Choose the stored .p12 certificate key. Must use provider type "file".'),
  '#default_value' => $config->get('apple_cert_key'),
  // Filters the available keys to a subset of all keys.
  '#key_filters' => [
    // Filters the available keys to those with the provider type set to "file".
    'provider' => 'file',
  ],
];

Creating a form element of type key_select will list all the keys in Drupal as a select element, allowing the user to select a single key. Setting the provider in #key_filters limits the keys shown to the user to keys with the provider type set to file.

When building the package, the module then requires the file path of the stored key file in order to build the package delivered to the user. This is handled as follows:

// Retrieve the PK Pass module settings configuration.
$settings = $this->configFactory->get('pkpass.settings');
// Get the key the user saved in the module configuration form.
$cert_key_id = $settings->get('apple_cert_key');
// Load the given key.
if ($cert_key = \Drupal::service('key.repository')->getKey($cert_key_id)) {
  // The key has been found. Generally, the key value is required, which would be retrieved with
  // $cert_key->getKeyValue(). However, in this
  // module, the filepath of the .p12 file is needed, rather
  // than the file contents. So the filepath is retrieved:
  $cert_path = NULL;
  /** @var \Drupal\key\Plugin\KeyProviderInterface $key_provider */
  $key_provider = $cert_key->getKeyProvider();
  // Ensure that the key provider is of type file.
  if ($key_provider instanceof KeyProviderBase && $key_provider->getPluginId() === 'file') {
    $provider_config = $key_provider->getConfiguration();
    // Determine the file location of the file.
    $file_location = $provider_config['file_location'] ?? NULL;
    if (!$file_location || !is_file($file_location)) {
      throw new \Exception('A certificate could not be loaded at the path ' . $file_location);
    }
  }
  else {
    throw new \Exception('The Apple Pass Certificate key must have a provider type of "file"');
  }
}
else {
  throw new \Exception('The Apple Pass Certificate Key could not be loaded.');
}

Summary

The key module is an excellent tool in Drupal for storing 3rd party keys in a secure manner that prevents them from falling into the hands of those nefarious actors inadvertently, which is regularly a component of a secure Drupal system. This module is well developed and has a great API for retrieval of keys for use with other modules. Happy Drupalling!

 

 

1 Note that key files should always be stored outside the webroot. I store my keys in the ../keys directory, with the directory permissions set to 500, and each key file permission set to 400 (your server may require 550 and 440 if the file owner and group are different).

2 If the keys directory is added to Git, the keys are exposed, defeating the purpose. If this ever happens accidentally, all keys should be regenerated.