Topics

Proposal for password hashing


Will Hopkins
 

EG:

I like the idea of supporting EL expressions so that users can specify their own password hashing algorithm, but I wonder if it wouldn't be better, or at least simpler, to support an interface for generate() and verify() password hashes -- I'm not sure we need EL logic, we just need an implementation of the algorithm.

Can someone make a suggestion for how to construct/document an EL expression that could be easily used by an idstore implementation to verify the hash for a password?

Ultimately, though, I think we should also support some specific algorithms. PBKDF2 is supported by the JDK, with hash functions ranging from SHA1 to SHA256. I would propose to support the following:

PBKDF2WithHmacSHA1
PBKDF2WithHmacSHA224
PBKDF2WithHmacSHA256
PBKDF2WithHmacSHA384
PBKDF2WithHmacSHA512

I have tried several of these in JDK8 and they all worked; however, I'd note that only SHA1 and SHA256 are required to be supported by implementations of Java SE, according to the algorithm names doc for JDK8.

We might also want to allow for parameters to be optionally specified: salt size, hash size, and iterations. We could specify that as, e.g., "PBKDF2WithHmac256_S24_H24_I1000". It might be better as separate parameters, but that makes the annotation more complicated, and other algorithms, e.g., Argon2, which is the current "best" algorithm, will need different parameters.

Or, we could keep it simple, and let the implementation choose the default parameters, and users that want more control can override the default. We can achieve interoperability by specifying the output format to include the parameters used for the hash, so that any implementation can figure that out and verify the hash.

Given that, we'll want to remove the HashEncoding attribute, and define the "standard" encoding, e.g.:

{Algorithm:interations:saltsize}base64(salt):base64(hash)

What do people think?

Will

-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803


Guillermo González de Agüero
 

Hi,

Sorry if I have missed something, but does this discussion originate from [1] or is it something that was talked about on the call?

I like the idea of a custom interface so users can define their own algorithms. While the spec should only support currently "secure" algorithms, there are unfortunately still a lot of legacy databases using MD5 hashed passwords.

Also, it would make it easir for users to debug and know what exactly the servers is doing. The most painful moment I have when configuring a new server to use a legacy database is knowing exactly how is it hashing the passwords. Standarizing an interface on it will simplify that process.

What's your idea for algorithms registration? @Named CDI beans implementing the new interface?

As for the parameters, what about just creating a new @HashEncoding annotation where everything can live? It's cleaner and can be reused on multiple identity stores. This was basically what Simon Bäurle proposed on [2]. It can contain the most common attributes and then an "additionalOptions" [3]. A syntax like the "{Algorithm:interations:saltsize}base64(salt):base64(hash)" you proposed will be IMHO difficult to read and will explode as new parameters are needed on future iterations.


Regards,

Guillermo González de Agüero

On Fri, Jul 7, 2017 at 10:45 PM, Will Hopkins <will.hopkins@...> wrote:
EG:

I like the idea of supporting EL expressions so that users can specify their own password hashing algorithm, but I wonder if it wouldn't be better, or at least simpler, to support an interface for generate() and verify() password hashes -- I'm not sure we need EL logic, we just need an implementation of the algorithm.

Can someone make a suggestion for how to construct/document an EL expression that could be easily used by an idstore implementation to verify the hash for a password?

Ultimately, though, I think we should also support some specific algorithms. PBKDF2 is supported by the JDK, with hash functions ranging from SHA1 to SHA256. I would propose to support the following:

PBKDF2WithHmacSHA1
PBKDF2WithHmacSHA224
PBKDF2WithHmacSHA256
PBKDF2WithHmacSHA384
PBKDF2WithHmacSHA512

I have tried several of these in JDK8 and they all worked; however, I'd note that only SHA1 and SHA256 are required to be supported by implementations of Java SE, according to the algorithm names doc for JDK8.

We might also want to allow for parameters to be optionally specified: salt size, hash size, and iterations. We could specify that as, e.g., "PBKDF2WithHmac256_S24_H24_I1000". It might be better as separate parameters, but that makes the annotation more complicated, and other algorithms, e.g., Argon2, which is the current "best" algorithm, will need different parameters.

Or, we could keep it simple, and let the implementation choose the default parameters, and users that want more control can override the default. We can achieve interoperability by specifying the output format to include the parameters used for the hash, so that any implementation can figure that out and verify the hash.

Given that, we'll want to remove the HashEncoding attribute, and define the "standard" encoding, e.g.:

{Algorithm:interations:saltsize}base64(salt):base64(hash)

What do people think?

Will

-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803



Will Hopkins
 

Hi Guillermo,

On 07/08/2017 02:49 AM, Guillermo González de Agüero wrote:
Hi,

Sorry if I have missed something, but does this discussion originate from [1] or is it something that was talked about on the call?
We discussed it on the call, where EL was proposed as a mechanism for dealing with the great variation in algorithms one might want to use. The original impetus to do something derives from https://github.com/javaee-security-spec/soteria/issues/48 and https://github.com/javaee-security-spec/soteria/issues/83.


I like the idea of a custom interface so users can define their own algorithms. While the spec should only support currently "secure" algorithms, there are unfortunately still a lot of legacy databases using MD5 hashed passwords.
Agreed -- we need to support working with old dbs, but shouldn't encourage new uses of (e.g.,) MD5.


Also, it would make it easir for users to debug and know what exactly the servers is doing. The most painful moment I have when configuring a new server to use a legacy database is knowing exactly how is it hashing the passwords. Standarizing an interface on it will simplify that process.
Perhaps -- I think you'd still need to look at what was actually in the DB to figure out what the algorithm needed to be ...


What's your idea for algorithms registration? @Named CDI beans implementing the new interface?
My idea was that there would be algorithms defined by the spec, and then the ability to provide your own algorithm via an EL expression, rather than a way to register new algorithms that would be known by the system. The ability to register algorithms would be great, but would take time to design, and we'd only want to support algorithms for which there is a standard Java SE name -- we should encourage all uses of encryption to go through JCE. (I know that, e.g., bouncy castle is widely used, in some cases because Java SE doesn't support particular algorithms/functions, but JCA is a good architecture and providers could in theory provide new algrorithms, whose names could be standardized.


As for the parameters, what about just creating a new @HashEncoding annotation where everything can live? It's cleaner and can be reused on multiple identity stores. This was basically what Simon Bäurle proposed on [2].
I thought about this, but I'm not sure there are really a lot of common parameters. PBKDF2 and bcrypt have a lot in common, but Argon2, the emerging standard, seems to be completely different. Also, there isn't enough time now to propose an annotation (or set of annotations) and get it reviewed before PFD. (There may be someone on the list who could whip something up quickly, but I"d need to spend some time understanding the details of how annotations work with inheritence, extensibility, etc., to feel like I'd done it right.)

It can contain the most common attributes and then an "additionalOptions" [3]. A syntax like the "{Algorithm:interations:saltsize}base64(salt):base64(hash)" you proposed will be IMHO difficult to read and will explode as new parameters are needed on future iterations.
That's mean to be the storage format, not the specification of the algorithm. If the storage format is well defined and carries all the parameters, then an implementation of a particular algorithm can deal with passwords previously hashed with different parameters. It's verbose and unattractive, but I believe it's deterministically parsable, and can support any algorithm:
  • Get the algorithm and its parameters by matching the first complete "{...}".  What's left is the salt and the hashed password.
  • Get the algorithm by splitting the algorithm string with ':' as the token separator. The first token is the algorithm name, the remaining tokens are the algorithm's parameters, which will vary depending on the algorithm, but should be in a standard order for any given algorithm.
  • Get the base64-encoded salt and password hash by splitting the remainder of the string using ':'.
  • base64-decode the salt and password hashes, which can be any length (i.e., any salt size or hash size value), and use the resulting lengths as input to the hashing algorithm.



On Fri, Jul 7, 2017 at 10:45 PM, Will Hopkins <will.hopkins@...> wrote:
EG:

I like the idea of supporting EL expressions so that users can specify their own password hashing algorithm, but I wonder if it wouldn't be better, or at least simpler, to support an interface for generate() and verify() password hashes -- I'm not sure we need EL logic, we just need an implementation of the algorithm.

Can someone make a suggestion for how to construct/document an EL expression that could be easily used by an idstore implementation to verify the hash for a password?

Ultimately, though, I think we should also support some specific algorithms. PBKDF2 is supported by the JDK, with hash functions ranging from SHA1 to SHA256. I would propose to support the following:

PBKDF2WithHmacSHA1
PBKDF2WithHmacSHA224
PBKDF2WithHmacSHA256
PBKDF2WithHmacSHA384
PBKDF2WithHmacSHA512

I have tried several of these in JDK8 and they all worked; however, I'd note that only SHA1 and SHA256 are required to be supported by implementations of Java SE, according to the algorithm names doc for JDK8.

We might also want to allow for parameters to be optionally specified: salt size, hash size, and iterations. We could specify that as, e.g., "PBKDF2WithHmac256_S24_H24_I1000". It might be better as separate parameters, but that makes the annotation more complicated, and other algorithms, e.g., Argon2, which is the current "best" algorithm, will need different parameters.

Or, we could keep it simple, and let the implementation choose the default parameters, and users that want more control can override the default. We can achieve interoperability by specifying the output format to include the parameters used for the hash, so that any implementation can figure that out and verify the hash.

Given that, we'll want to remove the HashEncoding attribute, and define the "standard" encoding, e.g.:

{Algorithm:interations:saltsize}base64(salt):base64(hash)

What do people think?

Will

-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803


-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803


Guillermo González de Agüero
 

Thanks Will for the detailed response.

El dom., 9 de julio de 2017 18:36, Will Hopkins <will.hopkins@...> escribió:
Hi Guillermo,


On 07/08/2017 02:49 AM, Guillermo González de Agüero wrote:
Hi,

Sorry if I have missed something, but does this discussion originate from [1] or is it something that was talked about on the call?
We discussed it on the call, where EL was proposed as a mechanism for dealing with the great variation in algorithms one might want to use. The original impetus to do something derives from https://github.com/javaee-security-spec/soteria/issues/48 and https://github.com/javaee-security-spec/soteria/issues/83.



I like the idea of a custom interface so users can define their own algorithms. While the spec should only support currently "secure" algorithms, there are unfortunately still a lot of legacy databases using MD5 hashed passwords.
Agreed -- we need to support working with old dbs, but shouldn't encourage new uses of (e.g.,) MD5.



Also, it would make it easir for users to debug and know what exactly the servers is doing. The most painful moment I have when configuring a new server to use a legacy database is knowing exactly how is it hashing the passwords. Standarizing an interface on it will simplify that process.
Perhaps -- I think you'd still need to look at what was actually in the DB to figure out what the algorithm needed to be ...
I usually know the hashing algorithm but then I have to figure out how to configure each server to do that, usually with little feedback on logs.

Having a dedicated standard interface makes easy to at least know where should I go to view what the server is doing.



What's your idea for algorithms registration? @Named CDI beans implementing the new interface?
My idea was that there would be algorithms defined by the spec, and then the ability to provide your own algorithm via an EL expression, rather than a way to register new algorithms that would be known by the system. The ability to register algorithms would be great, but would take time to design, and we'd only want to support algorithms for which there is a standard Java SE name -- we should encourage all uses of encryption to go through JCE. (I know that, e.g., bouncy castle is widely used, in some cases because Java SE doesn't support particular algorithms/functions, but JCA is a good architecture and providers could in theory provide new algrorithms, whose names could be standardized.

How would that EL expression work? Just a method expression to a method that takes a char[] as argument?


As for the parameters, what about just creating a new @HashEncoding annotation where everything can live? It's cleaner and can be reused on multiple identity stores. This was basically what Simon Bäurle proposed on [2].
I thought about this, but I'm not sure there are really a lot of common parameters. PBKDF2 and bcrypt have a lot in common, but Argon2, the emerging standard, seems to be completely different. Also, there isn't enough time now to propose an annotation (or set of annotations) and get it reviewed before PFD. (There may be someone on the list who could whip something up quickly, but I"d need to spend some time understanding the details of how annotations work with inheritence, extensibility, etc., to feel like I'd done it right.)
Understood. Certainly this is something you configure only once so it's not such a great deal.


It can contain the most common attributes and then an "additionalOptions" [3]. A syntax like the "{Algorithm:interations:saltsize}base64(salt):base64(hash)" you proposed will be IMHO difficult to read and will explode as new parameters are needed on future iterations.
That's mean to be the storage format, not the specification of the algorithm. If the storage format is well defined and carries all the parameters, then an implementation of a particular algorithm can deal with passwords previously hashed with different parameters. It's verbose and unattractive, but I believe it's deterministically parsable, and can support any algorithm:
  • Get the algorithm and its parameters by matching the first complete "{...}".  What's left is the salt and the hashed password.
  • Get the algorithm by splitting the algorithm string with ':' as the token separator. The first token is the algorithm name, the remaining tokens are the algorithm's parameters, which will vary depending on the algorithm, but should be in a standard order for any given algorithm.
  • Get the base64-encoded salt and password hash by splitting the remainder of the string using ':'.
  • base64-decode the salt and password hashes, which can be any length (i.e., any salt size or hash size value), and use the resulting lengths as input to the hashing algorithm.



On Fri, Jul 7, 2017 at 10:45 PM, Will Hopkins <will.hopkins@...> wrote:
EG:

I like the idea of supporting EL expressions so that users can specify their own password hashing algorithm, but I wonder if it wouldn't be better, or at least simpler, to support an interface for generate() and verify() password hashes -- I'm not sure we need EL logic, we just need an implementation of the algorithm.

Can someone make a suggestion for how to construct/document an EL expression that could be easily used by an idstore implementation to verify the hash for a password?

Ultimately, though, I think we should also support some specific algorithms. PBKDF2 is supported by the JDK, with hash functions ranging from SHA1 to SHA256. I would propose to support the following:

PBKDF2WithHmacSHA1
PBKDF2WithHmacSHA224
PBKDF2WithHmacSHA256
PBKDF2WithHmacSHA384
PBKDF2WithHmacSHA512

I have tried several of these in JDK8 and they all worked; however, I'd note that only SHA1 and SHA256 are required to be supported by implementations of Java SE, according to the algorithm names doc for JDK8.

We might also want to allow for parameters to be optionally specified: salt size, hash size, and iterations. We could specify that as, e.g., "PBKDF2WithHmac256_S24_H24_I1000". It might be better as separate parameters, but that makes the annotation more complicated, and other algorithms, e.g., Argon2, which is the current "best" algorithm, will need different parameters.

Or, we could keep it simple, and let the implementation choose the default parameters, and users that want more control can override the default. We can achieve interoperability by specifying the output format to include the parameters used for the hash, so that any implementation can figure that out and verify the hash.

Given that, we'll want to remove the HashEncoding attribute, and define the "standard" encoding, e.g.:

{Algorithm:interations:saltsize}base64(salt):base64(hash)

What do people think?

Will

-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803


-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803


Will Hopkins
 



On 07/09/2017 01:01 PM, Guillermo González de Agüero wrote:
Thanks Will for the detailed response.

El dom., 9 de julio de 2017 18:36, Will Hopkins <will.hopkins@...> escribió:
Hi Guillermo,


On 07/08/2017 02:49 AM, Guillermo González de Agüero wrote:
Hi,

Sorry if I have missed something, but does this discussion originate from [1] or is it something that was talked about on the call?
We discussed it on the call, where EL was proposed as a mechanism for dealing with the great variation in algorithms one might want to use. The original impetus to do something derives from https://github.com/javaee-security-spec/soteria/issues/48 and https://github.com/javaee-security-spec/soteria/issues/83.



I like the idea of a custom interface so users can define their own algorithms. While the spec should only support currently "secure" algorithms, there are unfortunately still a lot of legacy databases using MD5 hashed passwords.
Agreed -- we need to support working with old dbs, but shouldn't encourage new uses of (e.g.,) MD5.



Also, it would make it easir for users to debug and know what exactly the servers is doing. The most painful moment I have when configuring a new server to use a legacy database is knowing exactly how is it hashing the passwords. Standarizing an interface on it will simplify that process.
Perhaps -- I think you'd still need to look at what was actually in the DB to figure out what the algorithm needed to be ...
I usually know the hashing algorithm but then I have to figure out how to configure each server to do that, usually with little feedback on logs.

Having a dedicated standard interface makes easy to at least know where should I go to view what the server is doing.



What's your idea for algorithms registration? @Named CDI beans implementing the new interface?
My idea was that there would be algorithms defined by the spec, and then the ability to provide your own algorithm via an EL expression, rather than a way to register new algorithms that would be known by the system. The ability to register algorithms would be great, but would take time to design, and we'd only want to support algorithms for which there is a standard Java SE name -- we should encourage all uses of encryption to go through JCE. (I know that, e.g., bouncy castle is widely used, in some cases because Java SE doesn't support particular algorithms/functions, but JCA is a good architecture and providers could in theory provide new algrorithms, whose names could be standardized.

How would that EL expression work? Just a method expression to a method that takes a char[] as argument?
You'd need two char[] arguments (or maybe a char[] and a string), to represent both the caller-supplied password and the hashed password retrieved from the database. You'd want to pass in the entire string returned from the DB, so the algorithm could parse out the algorithm parameters, salt, etc.



As for the parameters, what about just creating a new @HashEncoding annotation where everything can live? It's cleaner and can be reused on multiple identity stores. This was basically what Simon Bäurle proposed on [2].
I thought about this, but I'm not sure there are really a lot of common parameters. PBKDF2 and bcrypt have a lot in common, but Argon2, the emerging standard, seems to be completely different. Also, there isn't enough time now to propose an annotation (or set of annotations) and get it reviewed before PFD. (There may be someone on the list who could whip something up quickly, but I"d need to spend some time understanding the details of how annotations work with inheritence, extensibility, etc., to feel like I'd done it right.)
Understood. Certainly this is something you configure only once so it's not such a great deal.


It can contain the most common attributes and then an "additionalOptions" [3]. A syntax like the "{Algorithm:interations:saltsize}base64(salt):base64(hash)" you proposed will be IMHO difficult to read and will explode as new parameters are needed on future iterations.
That's mean to be the storage format, not the specification of the algorithm. If the storage format is well defined and carries all the parameters, then an implementation of a particular algorithm can deal with passwords previously hashed with different parameters. It's verbose and unattractive, but I believe it's deterministically parsable, and can support any algorithm:
  • Get the algorithm and its parameters by matching the first complete "{...}".  What's left is the salt and the hashed password.
  • Get the algorithm by splitting the algorithm string with ':' as the token separator. The first token is the algorithm name, the remaining tokens are the algorithm's parameters, which will vary depending on the algorithm, but should be in a standard order for any given algorithm.
  • Get the base64-encoded salt and password hash by splitting the remainder of the string using ':'.
  • base64-decode the salt and password hashes, which can be any length (i.e., any salt size or hash size value), and use the resulting lengths as input to the hashing algorithm.



On Fri, Jul 7, 2017 at 10:45 PM, Will Hopkins <will.hopkins@...> wrote:
EG:

I like the idea of supporting EL expressions so that users can specify their own password hashing algorithm, but I wonder if it wouldn't be better, or at least simpler, to support an interface for generate() and verify() password hashes -- I'm not sure we need EL logic, we just need an implementation of the algorithm.

Can someone make a suggestion for how to construct/document an EL expression that could be easily used by an idstore implementation to verify the hash for a password?

Ultimately, though, I think we should also support some specific algorithms. PBKDF2 is supported by the JDK, with hash functions ranging from SHA1 to SHA256. I would propose to support the following:

PBKDF2WithHmacSHA1
PBKDF2WithHmacSHA224
PBKDF2WithHmacSHA256
PBKDF2WithHmacSHA384
PBKDF2WithHmacSHA512

I have tried several of these in JDK8 and they all worked; however, I'd note that only SHA1 and SHA256 are required to be supported by implementations of Java SE, according to the algorithm names doc for JDK8.

We might also want to allow for parameters to be optionally specified: salt size, hash size, and iterations. We could specify that as, e.g., "PBKDF2WithHmac256_S24_H24_I1000". It might be better as separate parameters, but that makes the annotation more complicated, and other algorithms, e.g., Argon2, which is the current "best" algorithm, will need different parameters.

Or, we could keep it simple, and let the implementation choose the default parameters, and users that want more control can override the default. We can achieve interoperability by specifying the output format to include the parameters used for the hash, so that any implementation can figure that out and verify the hash.

Given that, we'll want to remove the HashEncoding attribute, and define the "standard" encoding, e.g.:

{Algorithm:interations:saltsize}base64(salt):base64(hash)

What do people think?

Will

-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803


-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803

-- 
Will Hopkins | WebLogic Security Architect | +1.781.442.0310
Oracle Application Development
35 Network Drive, Burlington, MA 01803