Vault for storing secrets (plus Spring Boot & Mongodb)

So this is a mix of a couple of howtos. First, we have a simple Spring Boot app that retrieves some data from a MongoDB database that has authentication enabled to access it. Next, we will see how to store secrets in a Vault implementation and then programmatically access it (in our case the user ID and password to access the database).

Setup a Local Mongodb instance

As always my instructions are for Mac. Make sure you have Homebrew installed.

# install on Mac
brew tap mongodb/brew
brew install mongodb-community@7.0

#start mongo
mongod --dbpath ~/temp/mongodata/

# install on Mac

brew tap mongodb/brew

brew install mongodb-community@7.0

#start mongo

mongod --dbpath ~/temp/mongodata/

We need some sample data to play with. Let’s use some Spotify top hits data extracts available on Kaggle. Download it from https://www.kaggle.com/datasets/josephinelsy/spotify-top-hit-playlist-2010-2022

# lets import some sample data 
mongoimport --host localhost --db spotifyhitsdb --collection hits --type csv --file ~/Downloads/playlist_2010to2022.csv --headerline --upsert --jsonArray

1 2	# lets import some sample data mongoimport --host localhost --db spotifyhitsdb --collection hits --type csv --file ~/Downloads/playlist_2010to2022.csv --headerline --upsert --jsonArray

To ensure data made it into the database, let’s spin up the Mongo shell and retrieve a document from the collection ‘hits’.

mongosh

mongosh

At the shell, you can switch to the spotifyhitsdb and retrieve one random document

Install Hashicorp Vault

Why do we use a Vault? When building enterprise software, we face the challenge of storing secrets or other such sensitive information that are not open to everyone. For example, the user ID and password to our database should not be accessible to everyone. Nor should they be hard-coded into configuration files checked into your code repository or stored in plain text on servers, etc. We need a centralized service that can scale for distributed systems and be governed by enterprise security policies. This is where tools like HashiCorp Vault comes into play. In this blog, I use HashiCorp Vault but the concepts are the same – protect sensitive data to protect your systems.

brew tap hashicorp/tap
brew install hashicorp/tap/vault

1 2	brew tap hashicorp/tap brew install hashicorp/tap/vault

Next start a local Vault in development mode. It goes without saying that you will not use dev mode for production. In most enterprises, you will have InfoSec point you to a central Vault service and each team will not be spinning up one.

# start a local server
vault server -dev --dev-root-token-id="00000000-0000-0000-0000-000000000000"

# if using CLI
export VAULT_ADDR='http://127.0.0.1:8200'
export VAULT_TOKEN="00000000-0000-0000-0000-000000000000"

# start a local server

vault server -dev --dev-root-token-id="00000000-0000-0000-0000-000000000000"

# if using CLI

export VAULT_ADDR='http://127.0.0.1:8200'

export VAULT_TOKEN="00000000-0000-0000-0000-000000000000"

Store a Secret. In our case, it is the credentials to the MongoDB database.

vault kv put secret/gs-vault-config mongo.username=myuser mongo.password=mypwd

1	vault kv put secret/gs-vault-config mongo.username=myuser mongo.password=mypwd

How to turn ON Authentication for local Mongodb

Mongo provides multiple authentication mechanisms. For this blog, we will use SCRAM.

# switch to the admin db
use admin

# create the credentials within a particular database
db.createUser({user:"myuser", pwd: "mypwd", roles: [{role: "userAdminAnyDatabase", db: "admin"},{role: "readWriteAnyDatabase", db: "admin"}]})

# restart the database (and the shell after this command), but after updating mongod.conf file
db.adminCommand( { shutdown: 1 } )

# switch to the admin db

use admin

# create the credentials within a particular database

db.createUser({user:"myuser", pwd: "mypwd", roles: [{role: "userAdminAnyDatabase", db: "admin"},{role: "readWriteAnyDatabase", db: "admin"}]})

# restart the database (and the shell after this command), but after updating mongod.conf file

db.adminCommand( { shutdown: 1 } )

If starting mongodb using mongod.conf file then add the following entry to the config file. Edit /usr/local/etc/mongod.conf (if using homebrew the file could be at /opt/homebrew/etc/mongod.conf)

Next, restart mongodb with auth turned on

# start mongbdb with auth enabled
mongod --auth --dbpath ~/temp/mongodata/

# reconnect to the shell
mongosh --authenticationDatabase "admin" -u "myuser" -p

# start mongbdb with auth enabled

mongod --auth --dbpath ~/temp/mongodata/

# reconnect to the shell

mongosh --authenticationDatabase "admin" -u "myuser" -p

The Spring App

Check the VaultApplication for the example code that will access the Mongodb database and print sample data retrieved. To prove the credentials were retrieved we print the user ID (of course never do that in a real app). Code at – https://github.com/thomasma/vault

VaultConfiguration is used to retrieve and make the credentials available for the connection code that resides in class MongoClientConfiguration.

package com.example.vault;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;

import com.mongodb.MongoClientSettings.Builder;
import com.mongodb.MongoCredential;
import com.mongodb.ServerAddress;

@Configuration
public class MongoClientConfiguration extends AbstractMongoClientConfiguration {
    @Autowired
    private VaultConfiguration vaultConfiguration;

    @Override
    public String getDatabaseName() {
        return "spotifyhitsdb";
    }

    @Override
    protected void configureClientSettings(Builder builder) {
        builder
                .credential(MongoCredential.createCredential(vaultConfiguration.getUsername(), "admin",
                        vaultConfiguration.getPassword().toCharArray()))
                .applyToClusterSettings(settings -> {
                    settings.hosts(List.of(new ServerAddress("127.0.0.1", 27017)));
                });
    }
}

package com.example.vault;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.context.annotation.Configuration;

import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;

import com.mongodb.MongoClientSettings.Builder;

import com.mongodb.MongoCredential;

import com.mongodb.ServerAddress;

@Configuration

public class MongoClientConfiguration extends AbstractMongoClientConfiguration {

@Autowired

private VaultConfiguration vaultConfiguration;

@Override

public String getDatabaseName() {

return "spotifyhitsdb";

}

@Override

protected void configureClientSettings(Builder builder) {

builder

.credential(MongoCredential.createCredential(vaultConfiguration.getUsername(), "admin",

vaultConfiguration.getPassword().toCharArray()))

.applyToClusterSettings(settings -> {

settings.hosts(List.of(new ServerAddress("127.0.0.1", 27017)));

});

}

We looked at two primitive operations above – storing and accessing a secret. But that is one piece of a much larger puzzle called Secrets Management. We need to be able to delete secrets, rotate secrets (due to changes in workforce or just good security policies), secure secrets using the highest encryption policies, govern who (user or system) has access, secret standards, maintain an audit of who has accessed the secrets, integrate this into other authentication systems (such as AWS, Azure, GCP), and so much more. Managing secrets is a complex and extremely critical responsibility and therefore you will often see commercial products stepping in (sometimes built on top of open source).

One thought on “Vault for storing secrets (plus Spring Boot & Mongodb)”

Howard October 20, 2023 at 10:11 am

Nice post. The storing of secret credentials vexes me, however. The sooner we move towards security architectures where “who you are” is what matters, and not “what you know”, the more secure and robust integrations will be. We have been increasingly relying on managed/workload identities, and in some cases public keys, and moving away from secret credentials. Can’t wait for a world with no secrets (and no passwords)

Comments are closed.

{"Mat's Random Thoughts"}

Mathew's Tech Notes..