Sunday, December 4, 2011

Apache Shiro Part 3 - Cryptography

Besides securing web pages and managing access rights Apache Shiro does also basic cryptography tasks. The framework is able to:
  • encrypt and decrypt data,
  • hash data,
  • generate random numbers.

Shiro does not implement any cryptography algorithms. All calculations are delegated to Java Cryptography Extension (JCE) API. The main benefit of using Shiro instead of what is already present in Java is ease of use and secure defaults. Shiro crypto module is written in higher abstraction level and by default implements all known best practices.

This is third part of series dedicated to Apache Shiro. First part showed how to secure web application and add log in/log out functionality. Second part showed how to store user accounts in database and give users an option to authenticate themselves via PGP certificates.

This post begins with a short Shiro and JCE overview and continues with description of few useful conversion utilities. Following chapters explain random number generation, hashing and how to encrypt and decrypt data. The final chapter shows how to customize a cipher and how to create a new one.

Overview

Shiro cryptography module resides in org.apache.shiro.crypto package. It does not have manual, but fortunately all crypto classes are Javadoc heavy. Javadoc contains everything that would be written in manual.

Shiro relies heavily on java cryptography extension. You do not need to understand JCE to use Shiro. However, you need JCE basics to customize it or add new features to it. If you are not interested in JCE, skip to the next chapter.

JCE is a set of highly customizable APIs and their default implementation. It is provider-based. If the default implementation does not have what you need, you can easily install a new provider.

Each cipher, cipher option, hash algorithm or any other JCE feature has a name. JCE defines two sets of standard names for algorithms and algorithm modes. Those are available with any JDK. Any provider, for example Bouncy Castle, is free to extend the names sets with new algorithms and options.

Names are composed into so-called transformations strings which are used to look up needed objects. For example, Cipher.getInstance("DES/ECB/PKCS5Padding") returns DES cipher in ECB mode with PKCS#5 padding. Returned cipher usually requires further initialization, may not use safe defaults and is not thread safe.

Apache Shiro composes transformation strings, configures acquired objects and adds thread safety to them. Most importantly, it has easy to use API and adds higher level best practices that should be implemented anyway.

Encoding, Decoding and ByteSource

Crypto package encrypts, decrypts and hashes byte arrays (byte[]). If you need to encrypt or hash s string, you have to convert it to byte array first. Conversely, if you need to store hashed or encrypted value in text file or string database column, you have to convert it to string.

Text to Byte Array
Static class CodecSupport is able to convert the text to byte array and back. The method byte[] toBytes(String source) converts a string to byte array and the method String toString(byte[] bytes) converts it back.

Example
Use codec support to convert between text and byte array:
@Test
 public void textToByteArray() {
  String encodeMe = "Hello, I'm a text.";

  byte[] bytes = CodecSupport.toBytes(encodeMe);
  String decoded = CodecSupport.toString(bytes);

  assertEquals(encodeMe, decoded);
 }

Encode and Decode Byte Arrays
Conversion from byte array to string is called encoding. The reverse process is called decoding. Shiro provides two different algorithms:
  • Base64 implemented in class Base64,
  • Hexadecimal implemented in class Hex.

Both classes are static and both have encodeToString and decode utility methods available.

Examples
Encode a random array into its Hexadecimal representation, decode it and verify the result:
@Test
 public void testStaticHexadecimal() {
  byte[] encodeMe = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20};
  
  String hexadecimal = Hex.encodeToString(encodeMe);
  assertEquals("020406080a0c0e101214", hexadecimal);
  
  byte[] decoded = Hex.decode(hexadecimal);
  assertArrayEquals(encodeMe, decoded);
 }

Encode a random array into its Byte64 representation, decode it and verify the result:
@Test
 public void testStaticBase64() {
  byte[] encodeMe = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20};
  
  String base64 = Base64.encodeToString(encodeMe);
  assertEquals("AgQGCAoMDhASFA==", base64);
  
  byte[] decoded = Base64.decode(base64);
  assertArrayEquals(encodeMe, decoded);
 }

ByteSource
Cryptography package often returns an instance of ByteSource interface instead of byte array. Its implementation SimpleByteSource is a simple wrapper around byte array with additional encoding methods available:
  • String toHex() - returns Hexadecimal byte array representation,
  • String toBase64() - returns Base64 byte array representation,
  • byte[] getBytes() - returns wrapped byte array.

Examples
The test uses ByteSource to encode an array into its Hexadecimal representation. It then decodes it and verifies the result:
@Test
 public void testByteSourceHexadecimal() {
  byte[] encodeMe = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20};
  
  ByteSource byteSource = ByteSource.Util.bytes(encodeMe);
  String hexadecimal = byteSource.toHex();
  assertEquals("020406080a0c0e101214", hexadecimal);
  
  byte[] decoded = Hex.decode(hexadecimal);
  assertArrayEquals(encodeMe, decoded);
 }

Use Bytesource to encode an array into its Base64 representation. Decode it and verify the result:
@Test
 public void testByteSourceBase64() {
  byte[] encodeMe = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20};
  
  ByteSource byteSource = ByteSource.Util.bytes(encodeMe);
  String base64 = byteSource.toBase64();
  assertEquals("AgQGCAoMDhASFA==", base64);
  
  byte[] decoded = Base64.decode(base64);
  assertArrayEquals(encodeMe, decoded);
 }

Random Number Generator

Random number generator is composed of RandomNumberGenerator interface and its default implementation SecureRandomNumberGenerator.

The interface is fairly simple, it has only two methods:
  • ByteSource nextBytes() - generates a random fixed length byte source,
  • ByteSource nextBytes(int numBytes) - generates a random byte source with specified length.

The default implementation implements these two methods and provides some additional configuration:
  • setSeed(byte[] bytes) - custom seed configuration,
  • setDefaultNextBytesSize(int defaultNextBytesSize) - the length of nextBytes() output.

The seed is a number (byte array in fact) that initializes random number generator. It allows you to generate 'predictable random numbers'. Two instances of the same random generator initialized with the same seed always generate the same random numbers sequence. It is useful for debugging, but be very careful with it.

If you can, do not specify custom seed for cryptography. Use the default one. Unless you really know what you are doing, the attacker may be able to guess the custom one. That would beat all security purposes of random numbers.

Under the hood: SecureRandomNumberGenerator delegates random number generation to JCE SecureRandom implementation.

Examples
First example creates two random number generators and verifies whether they generate two different things:
@Test
 public void testRandomWithoutSeed() {
  //create random generators
  RandomNumberGenerator firstGenerator = new SecureRandomNumberGenerator();
  RandomNumberGenerator secondGenerator = new SecureRandomNumberGenerator();
  
  //generate random bytes
  ByteSource firstRandomBytes = firstGenerator.nextBytes();
  ByteSource secondRandomBytes = secondGenerator.nextBytes();
 
  //compare random bytes
  assertByteSourcesNotSame(firstRandomBytes, secondRandomBytes);
 }

Second example creates two random number generators, initializes them with the same seed and checks whether they generate the same expected 20 bytes long random array:
@Test
 public void testRandomWithSeed() {
  byte[] seed = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
  
  //create and initialize first random generator
  SecureRandomNumberGenerator firstGenerator = new SecureRandomNumberGenerator();
  firstGenerator.setSeed(seed);
  firstGenerator.setDefaultNextBytesSize(20);

  //create and initialize second random generator
  SecureRandomNumberGenerator secondGenerator = new SecureRandomNumberGenerator();
  secondGenerator.setSeed(seed);
  secondGenerator.setDefaultNextBytesSize(20);

  //generate random bytes
  ByteSource firstRandomBytes = firstGenerator.nextBytes();
  ByteSource secondRandomBytes = secondGenerator.nextBytes();
 
  //compare random arrays
  assertByteSourcesEquals(firstRandomBytes, secondRandomBytes);

  //following nextBytes are also the same
  ByteSource firstNext = firstGenerator.nextBytes();
  ByteSource secondNext = secondGenerator.nextBytes();

  //compare random arrays
  assertByteSourcesEquals(firstRandomBytes, secondRandomBytes);

  //compare against expected values
  byte[] expectedRandom = {-116, -31, 67, 27, 13, -26, -38, 96, 122, 31, -67, 73, -52, -4, -22, 26, 18, 22, -124, -24};
  assertArrayEquals(expectedRandom, firstNext.getBytes());
 }

Hashing

A hash function takes an arbitrary long data as an input and converts it to a smaller fixed length data. Hash function result is called hash. Hashing is one way operation. It is not possible to convert hash back to original data.

The most important thing to remember is: always store passwords hash instead of password itself. Never ever store it directly.

Shiro provides two hash related interfaces, both support two concepts necessary for secure password hashing: salting and hash iterations:
  • Hash - represents hash algorithm.
  • Hasher - use this to hash passwords.

A salt is a random array concatenated to the password before hashing. It is usually stored together with the password. Without salt, identical passwords would have the same hash. That would make password hacking much easier.

Specify a number of hash iterations to slow down the hash operation. The slower the operation, the more difficult it is to crack stored passwords. Use a lot of iterations.

Hash
Hash interface implementations compute hash functions. Shiro implements six standard hash functions: Md2, Md5, Sha1, Sha256, Sha384 and Sha512.

Each hash implementation extends from ByteSource. The constructor takes input data, salt and number of required iterations. Salt and iterations number are optional.

ByteSource interface methods return:
  • byte[] getBytes() - hash,
  • String toBase64() - hash in Base64 representation,
  • String toHex() - hash in Hexadecimal representation.

Following code computes Md5 hash of "Hello Md5" text with no salt:
@Test
 public void testMd5Hash() {
  Hash hash = new Md5Hash("Hello Md5");
  
  byte[] expectedHash = {-7, 64, 38, 26, 91, 99, 33, 9, 37, 50, -22, -112, -99, 57, 115, -64};
  assertArrayEquals(expectedHash, hash.getBytes());
  assertEquals("f940261a5b6321092532ea909d3973c0", hash.toHex());
  assertEquals("+UAmGltjIQklMuqQnTlzwA==", hash.toBase64());

  print(hash, "Md5 with no salt iterations of 'Hello Md5': ");
 }

Next snippet calculates 10 iterations of Sha256 with salt:
@Test
 public void testIterationsSha256Hash() {
  byte[] salt = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

  Hash hash = new Sha256Hash("Hello Sha256", salt, 10);
  
  byte[] expectedHash = {24, 4, -97, -61, 70, 28, -29, 85, 110, 0, -107, -8, -12, -93, -121, 99, -5, 23, 60, 46, -23, 92, 67, -51, 65, 95, 84, 87, 49, -35, -78, -115};
  String expectedHex = "18049fc3461ce3556e0095f8f4a38763fb173c2ee95c43cd415f545731ddb28d";
  String expectedBase64 = "GASfw0Yc41VuAJX49KOHY/sXPC7pXEPNQV9UVzHdso0=";
  
  assertArrayEquals(expectedHash, hash.getBytes());
  assertEquals(expectedHex, hash.toHex());
  assertEquals(expectedBase64, hash.toBase64());

  print(hash, "Sha256 with salt and 10 iterations of 'Hello Sha256': ");
 }

Compare iterations calculated by the framework and by the client code:
@Test
 public void testIterationsDemo() {
  byte[] salt = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
  
  //iterations computed by the framework 
  Hash shiroIteratedHash = new Sha256Hash("Hello Sha256", salt, 10);

  //iterations computed by the client code 
  Hash clientIteratedHash = new Sha256Hash("Hello Sha256", salt);
  for (int i = 1; i < 10; i++) {
   clientIteratedHash = new Sha256Hash(clientIteratedHash.getBytes());
  }
  
  //compare results
  assertByteSourcesEquals(shiroIteratedHash, clientIteratedHash);
 }

Under the hood: all concrete hash classes extend from SimpleHash which delegates hash computation to JCE MessageDigest implementation. If you wish to extend Shiro with another hash function, instance it directly. The constructor takes JCE message digest (hash) algorithm name as a parameter.

Hasher
Hasher works on top of hash functions and implements best practices related to salting. The interface has only one method:
  • HashResponse computeHash(HashRequest request)

Hash request provides byte source to be hashed and an optional salt. Hash response returns a hash and a salt. The response salt is not necessary the same as supplied salt. More importantly, it may not be the whole salt used for hashing operation.

Any hasher implementation is free to generate its own random salt. The default implementation does that only if the request contains null salt. Additionally, used salt may be composed of 'base salt' and 'public salt'. 'Public salt' is returned in the hash response.

To understand why it is done this way, you have to recall that salt is usually stored together with the password. The attacker with access to the database would have all information needed for brute-force attack.

Therefore, the 'public salt' is stored at the same place as the password and 'base salt' is stored elsewhere. The attacker then needs to get access to two different places.

Default hasher is configurable. You can specify base salt, number of iterations and hash algorithm to be used. Use hash algorithm name from any Shiro hash implementation. It also always returns public salt from the hash request. See the demo:
@Test
 public void fullyConfiguredHasher() {
  ByteSource originalPassword = ByteSource.Util.bytes("Secret");

  byte[] baseSalt = {1, 1, 1, 2, 2, 2, 3, 3, 3};
  int iterations = 10;
  DefaultHasher hasher = new DefaultHasher();
  hasher.setBaseSalt(baseSalt);
  hasher.setHashIterations(iterations);
  hasher.setHashAlgorithmName(Sha256Hash.ALGORITHM_NAME);
  
  //custom public salt
  byte[] publicSalt = {1, 3, 5, 7, 9};
  ByteSource salt = ByteSource.Util.bytes(publicSalt);
  
  //use hasher to compute password hash
  HashRequest request = new SimpleHashRequest(originalPassword, salt);
  HashResponse response = hasher.computeHash(request);
  
  byte[] expectedHash = {55, 9, -41, -9, 82, -24, 101, 54, 116, 16, 2, 68, -89, 56, -41, 107, -33, -66, -23, 43, 63, -61, 6, 115, 74, 96, 10, -56, -38, -83, -17, 57};
  assertArrayEquals(expectedHash, response.getHash().getBytes());
 }

If you need compare passwords or data check-sums, provide a 'public salt' back to the same hasher. It will reproduce the hash operation. The example uses Shiro DefaultHasher implementation:
@Test
 public void hasherDemo() {
  ByteSource originalPassword = ByteSource.Util.bytes("Secret");
  ByteSource suppliedPassword = originalPassword;
  Hasher hasher = new DefaultHasher();
  
  //use hasher to compute password hash
  HashRequest originalRequest = new SimpleHashRequest(originalPassword);
  HashResponse originalResponse = hasher.computeHash(originalRequest);
  
  //Use salt from originalResponse to compare stored password with user supplied password. We assume that user supplied correct password.
  HashRequest suppliedRequest = new SimpleHashRequest(suppliedPassword, originalResponse.getSalt());
  HashResponse suppliedResponse = hasher.computeHash(suppliedRequest);
  assertEquals(originalResponse.getHash(), suppliedResponse.getHash());
  
  //important: the same request hashed twice may lead to different results 
  HashResponse anotherResponse = hasher.computeHash(originalRequest);
  assertNotSame(originalResponse.getHash(), anotherResponse.getHash());
 }

Note: as the supplied public salt in the above example was null, default hasher generated new random public salt.

Encryption / Decryption

A cipher encrypts the data into ciphertext unreadable without a secret key. Ciphers are divided into two groups: symmetric and asymmetric. A symmetric cipher uses the same key for encryption and decryption. Asymmetric cipher uses two different keys, one is used for encryption and another for decryption.

Apache Shiro contains two symmetric ciphers: AES and Blowfish. Both are stateless and thus thread-safe. Asymmetric ciphers are not supported.

Both ciphers are able to generate random encryption key and both implement CipherService interface. The interface defines two encryption and two decryption methods. First group serves for encryption/decryption of byte arrays:
  • ByteSource encrypt(byte[] raw, byte[] encryptionKey),
  • ByteSource decrypt(byte[] encrypted, byte[] decryptionKey).

Second group encrypts/decrypts streams:
  • encrypt(InputStream in, OutputStream out, byte[] encryptionKey),
  • decrypt(InputStream in, OutputStream out, byte[] decryptionKey).

Next code snippet generates new key, encrypts secret message with AES cipher, decrypts it and compares original message with decryption result:
@Test
 public void encryptStringMessage() {
  String secret = "Tell nobody!";
  AesCipherService cipher = new AesCipherService();
  
  //generate key with default 128 bits size
  Key key = cipher.generateNewKey();
  byte[] keyBytes = key.getEncoded();
  
  //encrypt the secret
  byte[] secretBytes = CodecSupport.toBytes(secret);
  ByteSource encrypted = cipher.encrypt(secretBytes, keyBytes);
  
  //decrypt the secret
  byte[] encryptedBytes = encrypted.getBytes();
  ByteSource decrypted = cipher.decrypt(encryptedBytes, keyBytes);
  String secret2 = CodecSupport.toString(decrypted.getBytes());
  
  //verify correctness
  assertEquals(secret, secret2);
 }

Another snipped shows how to encrypt/decryption streams with Blowfish. Shiro ciphers do not close nor flush neither input nor output stream. You have to do it by yourself:
@Test
 public void encryptStream() {
  InputStream secret = openSecretInputStream();
  BlowfishCipherService cipher = new BlowfishCipherService();

  // generate key with default 128 bits size
  Key key = cipher.generateNewKey();
  byte[] keyBytes = key.getEncoded();

  // encrypt the secret
  OutputStream encrypted = openSecretOutputStream();
  try {
   cipher.encrypt(secret, encrypted, keyBytes);
  } finally {
   // The cipher does not flush neither close streams.
   closeStreams(secret, encrypted);
  }

  // decrypt the secret
  InputStream encryptedInput = convertToInputStream(encrypted);
  OutputStream decrypted = openSecretOutputStream();
  try {
   cipher.decrypt(encryptedInput, decrypted, keyBytes);
  } finally {
   // The cipher does not flush neither close streams.
   closeStreams(secret, encrypted);
  }

  // verify correctness
  assertStreamsEquals(secret, decrypted);
 }

If you encrypt the same text with the same key twice, you will get two different encrypted texts:
@Test
 public void unpredictableEncryptionProof() {
  String secret = "Tell nobody!";
  AesCipherService cipher = new AesCipherService();

  // generate key with default 128 bits size
  Key key = cipher.generateNewKey();
  byte[] keyBytes = key.getEncoded();

  // encrypt two times
  byte[] secretBytes = CodecSupport.toBytes(secret);
  ByteSource encrypted1 = cipher.encrypt(secretBytes, keyBytes);
  ByteSource encrypted2 = cipher.encrypt(secretBytes, keyBytes);

  // verify correctness
  assertArrayNotSame(encrypted1.getBytes(), encrypted2.getBytes());
 }

Both previous examples used Key generateNewKey() method to generate keys. Use the method setKeySize(int keySize) to override the default key size (128 bits). Alternatively, the keyBitSize parameter of the method Key generateNewKey(int keyBitSize) specifies a key size in bits.

Some ciphers support only some key sizes. For example, AES supports only 128, 192, and 256 bits log keys:
@Test(expected=RuntimeException.class)
 public void aesWrongKeySize() {
  AesCipherService cipher = new AesCipherService();
  
  //The call throws an exception. Aes supports only keys of 128, 192, and 256 bits.
  cipher.generateNewKey(200);
 }

 @Test
 public void aesGoodKeySize() {
  AesCipherService cipher = new AesCipherService();
  //aes supports only keys of 128, 192, and 256 bits
  cipher.generateNewKey(128);
  cipher.generateNewKey(192);
  cipher.generateNewKey(256);
 }

As far as basics go, this is it. You do not need more to encrypt and decrypt sensitive data in your applications.

Update: I was overly optimistic here. Learning more is always useful, especially if you are handling sensitive data. This method is mostly, but not entirely secure. Both the problem and the solution are described in my other post.

Encryption / Decryption - Advanced

Previous chapter showed how to encrypt and decrypt some data. This chapter shows little bit more about how Shiro encryption works and how to customize it. It also shows how to easily add a new cipher if the standard two are not suitable for you.

Initialization Vector
Initialization vector is randomly generated byte array used during ecryption. The cipher that uses initialization vector is less predictable and thus harder to decrypt for an attacker.

Shiro automatically generates initialization vector and uses it to encrypt the data. The vector is then concatenated with encrypted data and returned to client code. You can turn it off by calling setGenerateInitializationVectors(false) on the cipher. The method is defined on JcaCipherService class. Both default encryption classes extend it.

Initialization vector size is encryption algorithm specific. If the default size (128 bits) does not work, use the method setInitializationVectorSize to customize it.

Random Generator
Turning off an initialization vector does not necessary mean that the cipher becomes predictable. Both Blowfish and AES have an element of randomness in them.

Following example turns off the initialization vector, but encrypted texts are still different:
@Test
 public void unpredictableEncryptionNoIVProof() {
  String secret = "Tell nobody!";
  AesCipherService cipher = new AesCipherService();
  cipher.setGenerateInitializationVectors(false);

  // generate key with default 128 bits size
  Key key = cipher.generateNewKey();
  byte[] keyBytes = key.getEncoded();

  // encrypt two times
  byte[] secretBytes = CodecSupport.toBytes(secret);
  ByteSource encrypted1 = cipher.encrypt(secretBytes, keyBytes);
  ByteSource encrypted2 = cipher.encrypt(secretBytes, keyBytes);

  // verify correctness
  assertArrayNotSame(encrypted1.getBytes(), encrypted2.getBytes());
 }

It is possible to customize or turn off the randomness. However, never ever do it in a production code. The randomness is absolute necessity for secure data encryption.

Both Shiro encryption algorithms extend from JcaCipherService class. The class have setSecureRandom(SecureRandom secureRandom) method. Secure random is standard java JCE random number generator. Extend it to create own implementation and pass it to the cipher.

Our ConstantSecureRandom implementation of SecureRandom always returns zero. We supplied it to the cipher and turned off the initialization vector to create an unsecure predictable encryption:
@Test
 public void predictableEncryption() {
  String secret = "Tell nobody!";
  AesCipherService cipher = new AesCipherService();
  cipher.setSecureRandom(new ConstantSecureRandom());
  cipher.setGenerateInitializationVectors(false);

  // define the key
  byte[] keyBytes = {5, -112, 36, 113, 80, -3, -114, 77, 38, 127, -1, -75, 65, -102, -13, -47};

  // encrypt first time
  byte[] secretBytes = CodecSupport.toBytes(secret);
  ByteSource encrypted = cipher.encrypt(secretBytes, keyBytes);

  // verify correctness, the result is always the same
  byte[] expectedBytes = {76, 69, -49, -110, -121, 97, -125, -111, -11, -61, 61, 11, -40, 26, -68, -58};
  assertArrayEquals(expectedBytes, encrypted.getBytes());
 }

Constant secure random implementation is long and uninteresting. It is available on Github.

Custom Cipher
Out of the box Shiro provides only Blowfish and AES encryption methods. The framework does not implement its own algorithms. Instead, it delegates the encryption to JCE classes.

Shiro provides only secure defaults and easier API. This design makes it possible to extend Shiro with any JCE block cipher.

Block ciphers encrypt messages per blocks. All blocks have equal fixed size. If the last block is too short, a padding is added to make it the same size as all other blocks. Each block is encrypted and combined with previously encrypted blocks.

Therefore, you have to configure:

Encryption Method
A custom cipher extends a DefaultBlockCipherService class. The class has only one constructor with one parameter: algorithm name. You may supply any JCE compatible algorithm name.

For example, this is source code of Shiro AES cipher:
public class AesCipherService extends DefaultBlockCipherService {

    private static final String ALGORITHM_NAME = "AES";

    public AesCipherService() {
        super(ALGORITHM_NAME);
    }

}

AES does not need to specify no other encryption parameter (block size, padding, encryption method). Defaults are good enough for AES.

Block Size
Default block cipher service has two methods for block size customization. The method setBlockSize(int blockSize) works only for byte array encoding and decoding. The method setStreamingBlockSize(int streamingBlockSize) works only for stream encoding and decoding.

The value 0 means that the default algorithm specific block size will be used. This is the default value.

Block cipher block size is very algorithm-specific. Selected encryption algorithm may not work with an arbitrary block size:
@Test(expected=CryptoException.class)
 public void aesWrongBlockSize() {
  String secret = "Tell nobody!";
  AesCipherService cipher = new AesCipherService();
  // set wrong block size 
  cipher.setBlockSize(200);

  // generate key with default 128 bits size
  Key key = cipher.generateNewKey();
  byte[] keyBytes = key.getEncoded();

  // encrypt the secret
  byte[] secretBytes = CodecSupport.toBytes(secret);
  cipher.encrypt(secretBytes, keyBytes);
 }

Padding
Use the method setPaddingScheme(PaddingScheme paddingScheme) to specify byte array encryption and decryption padding. The method setStreamingPaddingScheme( PaddingScheme paddingScheme) specifies stream encryption and decryption padding.

The enumeration PaddingScheme represents all typical padding schemes. Not all of them are available by default, you might have to install custom JCE provider to use them.

The value null means that the default algorithm specific padding will be used. This is the default value.

If you need a padding not included in the PaddingScheme enumeration, use either setPaddingSchemeName or setStreamingPaddingSchemeName methods. These methods take a string with padding scheme name as a parameter. They are less type-safe but more flexible than the above ones.

Padding is very algorithm-specific. Selected encryption algorithm may not work with an arbitrary padding:
@Test(expected=CryptoException.class)
 public void aesWrongPadding() {
  String secret = "Tell nobody!";
  BlowfishCipherService cipher = new BlowfishCipherService();
  // set wrong block size 
  cipher.setPaddingScheme(PaddingScheme.PKCS1);

  // generate key with default 128 bits size
  Key key = cipher.generateNewKey();
  byte[] keyBytes = key.getEncoded();

  // encrypt the secret
  byte[] secretBytes = CodecSupport.toBytes(secret);
  cipher.encrypt(secretBytes, keyBytes);
 }


Operation Mode
Operation mode specifies how are blocks chained (combined) together. As it was with a padding scheme, you might use either an OperationMode enumeration or a string to supply them.

Be careful, not each operation mode might be available. Additionally, they are not born equal. Some chaining modes are less safe than others. The default Cipher Feedback operation mode is both safe and available on all JDK environments.

Methods to set the operation mode for byte array encryption and decryption:
  • setMode(OperationMode mode)
  • setModeName(String modeName)

Methods to set the operation mode for stream encryption and decryption:
  • setStreamingMode(OperationMode mode)
  • setStreamingModeName(String modeName)

Exercise - Decrypt Openssl
Suppose that an application sends data encrypted with Linux openssl command. We know both hexadecimal representation of the key and command used to encrypt the data:
  • The key: B9FAB84B65870109A6E8707BC95151C245BF18204C028A6A.
  • The command: openssl des3 -base64 -p -K <secret key> -iv <initialization vector>.

Each message contains both hexadecimal representation of the initialization vector and base64 encoded encrypted message.

Sample message:
  • The initialization vector: F758CEEB7CA7E188.
  • The message: GmfvxhbYJbVFT8Ad1Xc+Gh38OBmhzXOV.

Generate Sample With OpenSSL
The sample message was encrypted with the command:
#encrypt "yeahh, that worked!" 
echo yeahh, that worked! | openssl des3 -base64 -p -K B9FAB84B65870109A6E8707BC95151C245BF18204C028A6A -iv F758CEEB7CA7E188

Use OpenSSL option -P to generate either a secret key or a random initial vector.

Solution
First, we have to find out algorithm name, padding and operation mode. Fortunately, all three are available in OpenSSL documentation. Des3 is an alias for triple DES encryption algorithm in CBC mode and OpenSSL uses PKCS#5 padding.

Cipher-block chaining (CBC) requires an initialization vector of the same size as the block size. Triple DES requires 64 bit long blocks. Java JCE uses "DESede" algorithm name for Triple DES.

Our custom cipher extends and configures DefaultBlockCipherService:
public class OpensslDes3CipherService extends DefaultBlockCipherService {

  public OpensslDes3CipherService() {
   super("DESede");
   setMode(OperationMode.CBC);
   setPaddingScheme(PaddingScheme.PKCS5);
   setInitializationVectorSize(64);
  }
  
 }

Shiro cipher decrypt method expects two input byte arrays, ciphertext and key. Ciphertext should contain both initialization vector and encrypted cipher text. Therefore, we have to combine them together before we try to decrypt the message. The method combine combines two arrays into one:
private byte[] combine(byte[] iniVector, byte[] ciphertext) {
  byte[] ivCiphertext = new byte[iniVector.length + ciphertext.length];

  System.arraycopy(iniVector, 0, ivCiphertext, 0, iniVector.length);
  System.arraycopy(ciphertext, 0, ivCiphertext, iniVector.length, ciphertext.length);

  return ivCiphertext;
 }

The actual decryption looks as usually:
@Test
 public void opensslDes3Decryption() {
  String hexInitializationVector = "F758CEEB7CA7E188";
  String base64Ciphertext = "GmfvxhbYJbVFT8Ad1Xc+Gh38OBmhzXOV";
  String hexSecretKey = "B9FAB84B65870109A6E8707BC95151C245BF18204C028A6A";

  //decode secret message and initialization vector
  byte[] iniVector = Hex.decode(hexInitializationVector);
  byte[] ciphertext = Base64.decode(base64Ciphertext);

  //combine initialization vector and ciphertext together
  byte[] ivCiphertext = combine(iniVector, ciphertext);
  
  //decode secret key
  byte[] keyBytes = Hex.decode(hexSecretKey);

  //initialize cipher and decrypt the message
  OpensslDes3CipherService cipher = new OpensslDes3CipherService();
  ByteSource decrypted = cipher.decrypt(ivCiphertext, keyBytes);
  
  //verify result
  String theMessage = CodecSupport.toString(decrypted.getBytes());
  assertEquals("yeahh, that worked!\n", theMessage);
 }


End

This part of Apache Shiro tutorial covered cryptography features available in 1.2 version. All used examples are available on Github.

30 comments:

Les Hazlewood said...

Meri,

You've outdone yourself! This is truly outstanding and an amazing resource for Shiro crypto concepts. Fantastic job!!!

Thanks once again for sharing with everyone - I'm sure this will help so many people!

Best,

Les

Meri said...

Hi Les,

thank you :).

Meri

Anonymous said...

Awesome! Awesome tutorial! Great job!

Anonymous said...

this is really superb

Anonymous said...

Thanks for these useful docs, while no docs are available on Apache Shiro's site.

inanme said...

Hi Meri,

Actually i could not figure out what you chose to write about this library(I am not saying shiro is good or bad). as i understand it is simple wrapper around JCE. Your other articles are much more sophisticated compared to this one.

gautham said...

Generally whenever you apply a hash algorithm to an certain amount of data, such as binary file. the input result is converts to hash. this hash be a fixed size. learn more in bitcoin online course

gautham said...

Cryptography process involves in the process of concealing all the contents of the system. there are mainly two types of cryptographic algorithms they are symmetric and assymetric. learn more on these concepts in blockchain training

gautham said...

to become a cyber security expert cyber security training is needed from scatch

gautham said...

this is the best post I have seen hadoop certification

svrtechnologies said...

Whatever we gathered information from the blogs, we should implement that in practically then only we can understand that exact thing clearly, but it’s no need to do it, because you have explained the concepts very well. It was crystal clear, keep sharing.. cyber security training online

Jason Miler said...

NapBots offers mathematical crypto trading robots that are powered by AI and maker knowing engines. It concentrates on short, tool or long-term horizons. This technique enables us to minimize functional dangers and also to achieve a very durable backtests with limited drawdowns. Bitcoin investment

Oliver said...

Trading Directory is an experienced and reliable Forex Trading company. Here We compare online brokers using transparent 10 point checklists, so you can easily compare the most important factors in one glance.

Unknown said...

Market Live Rates Provides You With Honest Reviews And Strategies For Each Website. Our Site Is Updated Frequently And New Strategies Are Tested Daily. Be Sure To Check Our Facebook Page As Well As Our Youtube To Stay In Touch With Market Live Rates Latest News And Updates!

Oliver said...

View detailed stock quotes and interactive charts for Uavs Stocktwits. The Current value section displays the Uavs Stocktwits price over time as well as the value of the stock in USD. Hovering over the current values will display the price movement in % points and clicking on the symbols will bring you to the relevant quote page. Use the analysis tools on top of this chart for real-time gathering and comparison of economic data about Uavs Stocktwits.

Maria Yang said...

No One Knows How Many Links Are Required To Rank On The First Page Of Google, But It's Safe To Say That There Are A Lot. We Will Do All The Work For You, Including The Tedious Advanced Seo Techniques For Crypto Brokers Process, For A Reasonable Price. We'll Make Sure That You Have A Complete List Of Links, Both Old And New, To Keep Your Site's Rankings High.

George said...

Get all of the Aces Fresenius Updates you need to succeed in investing and trading on the Live, Real-Time Stock Market Overview. Be one of the first to know if Aces Fresenius has increased in value or if other stocks are trending up or down. Live notifications will tell you about all of these things in a clear and concise manner so that you can make the right decisions on your next investment in Aces Fresenius.

Online Edu Tech said...

This is a really fascinating and useful blog. I've read a lot of blogs lately, but your writing style is quite distinct and insightful. If you've read my articles, please go to the next step.

Cyber Security Course

Online Edu Tech said...

This is excellent news for me; thank you for sharing it, buddy!

Cyber Security Interview Questions

careerera said...
This comment has been removed by the author.
Market Tutor said...

Markettutors is a financial educational institutions which provide knowledge and training to traders , investors, students in financial market, stock market & accounting sector. They provide long and short term tarining to job oriented employees. They provide knowledge in financial sector related to Share Market, Debt Market, Derivatives Market, Commodities Market, Technical Analysis, Fundamental Analysis, Research Analyst, Investment Advisors, Mutual Funds, Insurance Sector and Banking Sector from basics to advance.They also help the people in clearing various exams and certifications from SEBI, NISM, IRDA and NCFM related to the stock or financial market.

contact@markettutors.in
8989743287

Online Edu Tech said...

Great Blog. I enjoyed reading this articles. Agile Interview Questions

Chaukor Studio Ghaziabad said...

Great Blog. Love reading this blog. Best Interior Designers In Noida

advancewellness said...

This is a wonderful Blog!! I loved reading it. Thanks for sharing all these information, it’s so appreciated!!!

PRP for hair loss in Ghaziabad

We are offering the best PRP for hair loss in Ghaziabad, PRP (platelet-rich plasma) therapy for hair loss is a three-step medical procedure in which blood is extracted, processed, and injected into the scalp.
PRP treatment in Ghaziabad is now easy and affordable.
To know more visit to - www.advancedwellness.in/prp-treatment/prp-for-hair-loss

AFM Logistics said...
This comment has been removed by the author.
Anonymous said...
This comment has been removed by the author.
Anonymous said...

You have amazing writing skills and you display them in every article. Keep it going!
Cyber Security Bootcamp

Anonymous said...

Thanks for sharing your informative blog with us.
If you are looking for online courses then visit our services Selenium Automated Testing

marry said...

This post was incredibly educational and beneficial, in my opinion. I spent a lot of time trying to share the spacebar counter article with you. I appreciate you sharing it. Click here spacebar click counter.

disqus said...

Have you ever considered about adding a little bit more than just your articles?
I mean, what you say is fundamental and all.
However think about if you added some great graphics or videos to give your posts more, « pop »!
Your content is excellent but with pics and videos, this blog could undeniably be one of the most beneficial in its niche.
Wonderful blog!
Feel free to surf to my web page :
diigo
wikidot
stageit
gta5-mods
bibrave
perchcms
plimbi
ekademia
yoomark

Post a Comment