Data Security Opening
As engineers, we must make data security a top priority, especially with sensitive user data being a prime target for hackers. Personal data such as identification numbers, email addresses, and phone numbers are particularly vulnerable to attacks. To protect this data, it is crucial that we do not store it in plaintext format. Instead, we should first encrypt the data before storing it in the database. This approach ensures that even if the data is compromised, hackers will not be able to access the sensitive information.
There are two types of data that we need to manage:
- Data that we do not need to know its original form, such as passwords. When it comes to user passwords, we only need to compare the hashed result of the user’s input with the data stored in the database.
- Data that we need to know its original information, such as identification numbers, email addresses, social security numbers, and so on.
By understanding the above goals, we can choose the appropriate way to secure data, whether by using Encoding, Encryption, or Hashing. Let’s discuss the differences between these three methods.
Encoding
Encoding is the process of changing data into another format using a specific algorithm. The algorithm used is widely known, so to restore the data to its original form, we only need to use the same algorithm. There is no need to use a key when decoding.
Some commonly used encoding algorithms are ASCII, Unicode, URL Encoding, Base64, etc. The most commonly used algorithm is Base64.
Encryption
Encryption is the process of scrambling readable data into unreadable text, also known as ciphertext. During the encryption process, cryptographic keys are required. As long as third parties or attackers do not have the cryptographic key, the ciphertext cannot be decrypted, ensuring that the stored value remains secure.
There are two types of encryption: symmetric encryption and asymmetric encryption.
A. Symmetric Encryption
Symmetric Encryption is encryption that uses only one key for both encryption and decryption. Therefore, the sender who encrypts the data must provide the key to the person who will decrypt it. Some commonly used algorithms for symmetric encryption are Caesar, Blowfish, and Advanced Encryption Standard or AES.
B. Asymmetric Encryption
For this type of encryption, two different related keys are required, called public and private keys. These two keys have different functions, with the public key used for encryption and the private key used for decryption. An example of an algorithm used for asymmetric encryption is the Rivest-Shamir-Adleman (RSA) algorithm.
Hashing
Unlike encoding and encryption, where the initial value can be restored, any plaintext data that is hashed cannot be restored to its original value. This is because hashing is known as a one-way function. Examples of algorithms used for hashing include MD5, SHA-1, SHA-2, SHA-3, RIPEMD-160, NTLM, and LANMAN.
Data Security Techniques
We can use encryption to ensure data security. Encryption requires an algorithm, key, and value to create ciphertext.
It is recommended not to store the key used for encryption in a single server. We can use Key Management Services (KMS) to store the key. Examples of KMS that we can use include Vault, AWS KMS, and GCP KMS.
I will try to write about my experience to rewrite this rails library https://github.com/ankane/lockbox to go language version.
In general, the steps to secure data (encrypt) are as follows:
- Input the data into the application.
- Generate a random key.
- Encrypt the generated random key using a KMS key and save it as an encrypted_kms_key.
- Encrypt the user input using the unencrypted random key and save it as ciphertext.
- To facilitate searching, the final step is to create a blind index from the user-inputted value.
This process to encrypt data
For the process of decrypting data:
- We need to decrypt the encrypted_kms_key to get the random key in plain text.
- After successfully obtaining the random key, we can decrypt the ciphertext using the random key. The final result will be the plain text input from the user.
Let’s create a sample application in Go. In this case, we will be using AWS KMS. Here, we will create a simple library as an example to make this application work.
First, let’s create a struct that will function for decrypting and encrypting.
type Kms interface {
Decrypt(ciphertext string, encryptionContext map[string]*string) (plainText []byte, err error)
Encrypt(message []byte, encryptionContext map[string]*string) (ciphertext string, err error)
}
Let’s break down the code above. It defines an interface called KmsAws with two methods: Decrypt() and Encrypt().
The Decrypt() method takes ciphertext and encryptionContext as input, and returns the plainText response as a byte array ([]byte). The Encrypt() method takes a message, which is the plaintext to be encrypted in the form of []byte, and encryptionContext as input. The output of this function is a string of ciphertext.
The KmsAws interface is used to perform encryption and decryption operations on data using the Key Management Service (KMS) in AWS (Amazon Web Services).
Next, we create a struct to implement the above interface.
type kmsAwsCtx struct {
Session *session.Session
SvcKms *kms.KMS
KmsKeyID string
}
type KmsAwsConfig struct {
AwsAccessKeyID string
AwsSecretAccessKey string
KmsKeyID string
Region string
}
The implementation of the interface is done in the kmsAwsCtx structure that stores the configuration and status of KMS. kmsAwsCtx consists of three fields, namely Session which represents the AWS session, SvcKms which represents the KMS service, and KmsKeyID which represents the ID of the KMS key to be used.
We also need to add the required configuration to create the session that we will use to encrypt and decrypt on the AWS service. This line defines the KmsAwsConfig structure that stores the configuration required to create an kmsAwsCtx instance. The stored configuration includes AwsAccessKeyID, AwsSecretAccessKey, KmsKeyID, and Region.
Then, we will create a function that implements the Kms interface with the following code:
func NewKmsAws(config *KmsAwsConfig) (Kms, error) {
kmsAws := kmsAwsCtx{}
var err error
kmsAws.Session, err = session.NewSession(&aws.Config{
Region: aws.String(config.Region),
Credentials: credentials.NewStaticCredentials(config.AwsAccessKeyID, config.AwsSecretAccessKey, ""),
})
if err != nil {
fmt.Println(err.Error())
return nil, err
}
kmsAws.SvcKms = kms.New(kmsAws.Session)
kmsAws.KmsKeyID = config.KmsKeyID
return &kmsAws, nil
}
The following code defines the KmsAws interface with two methods: Decrypt() and Encrypt(). The Decrypt() method takes in a ciphertext and an encryptionContext as input, and returns the plaintext response as a byte slice. The Encrypt() method takes in a message, which is the plaintext to be encrypted as a byte slice, as well as an encryptionContext for providing additional parameters during encryption. The output of this method is a ciphertext string. The KmsAws interface is used for encrypting and decrypting data using the Key Management Service (KMS) on AWS.
The interface is then implemented in the kmsAwsCtx struct, which stores the KMS configuration and status. kmsAwsCtx has three fields: Session, which represents the AWS session, SvcKms, which represents the KMS service, and KmsKeyID, which represents the ID of the KMS key to be used.
The code also includes the configuration needed to create the session that will be used to perform encryption and decryption on the AWS service. The KmsAwsConfig struct is defined to store the necessary configuration for creating an instance of kmsAwsCtx. The stored configuration includes AwsAccessKeyID, AwsSecretAccessKey, KmsKeyID, and Region.
Finally, a function is created to implement the KmsAws interface. The NewKmsAws() function takes a KmsAwsConfig parameter containing AWS configuration details such as AwsAccessKeyID, AwsSecretAccessKey, KmsKeyID, and Region. Within the function, an instance of kmsAwsCtx is created using the provided configuration, and an AWS session is created using aws.Config and session.NewSession(). Then, an instance of the KMS service is created using kms.New() and the provided configuration.
Finally, the implementation for the Decrypt() and Encrypt() methods are added to the code.
func (c *kmsAwsCtx) Decrypt(chipperText string, encryptionContext map[string]*string) (plainText []byte, err error) {
chipperTextArr := strings.Split(chipperText, `:`)
if len(chipperTextArr) < 2 {
return nil, errors.New(`Invalid Chippertext`)
}
chipperText = chipperTextArr[1]
ciphertextBlob, err := base64.StdEncoding.DecodeString(chipperText)
if err != nil {
fmt.Println(err.Error())
return nil, err
}
inputDecrypt := &kms.DecryptInput{
CiphertextBlob: ciphertextBlob,
EncryptionContext: encryptionContext,
}
respDecrypt, err := c.SvcKms.Decrypt(inputDecrypt)
if err != nil {
fmt.Println(err)
return nil, err
}
return respDecrypt.Plaintext, nil
}
In the Decrypt() method, it is used to perform decryption on a ciphertext. This ciphertext must have the format v1:base64-encoded-ciphertext. In this method, the given ciphertext will be decoded from base64, then used to call the Decrypt() function from the KMS service with the parameters CiphertextBlob and EncryptionContext. The result of the Decrypt() function is plaintext that will be returned.
func (c *kmsAwsCtx) Encrypt(message []byte, encryptionContext map[string]*string) (cipherText string, err error) {
inputEncrypt := &kms.EncryptInput{
KeyId: aws.String(c.KmsKeyID),
Plaintext: []byte(message),
EncryptionContext: encryptionContext,
}
respEncrypt, err := c.SvcKms.Encrypt(inputEncrypt)
if err != nil {
return ``, err
}
dst := make([]byte, base64.StdEncoding.EncodedLen(len(respEncrypt.CiphertextBlob)))
base64.StdEncoding.Encode(dst, respEncrypt.CiphertextBlob)
cipherText = `v1:` + string(dst)
return cipherText, nil
}
The Encrypt() method is used to encrypt a plaintext. In this method, the plaintext provided will be encrypted using the Encrypt() function from the KMS service with parameters KeyId, Plaintext, and EncryptionContext. The result of the Encrypt() function is ciphertext which is then encoded in the v1:base64-encoded-ciphertext format and returned.
Gotcha, that library can use to decrypt ciphertext from rails version, and vice versa, you can decrypt on golang and encrypt with rails version as long the configuration is similar between rails and go.
we have created a sample how to use the Key Management Service (KMS) from Amazon Web Services (AWS) to encrypt and decrypt data.
Overall, this sample application demonstrates how to use the KMS service from AWS to protect sensitive data by encrypting and decrypting it using a secure key.
In the next part, I will give an example about implementation this library and writing about generate random key on golang version.
Abi Wardani 