Discussion:
[asio-users] adding new functionality to the boost::asio::ssl context (new method for passing the private key)
Niculae Laurentiu
2012-11-22 15:21:05 UTC
Permalink
Hi, 

I'm using boost::asio::ssl and what I'm trying to achieve is to pass the SSL private key for a certificate to the boost::asio::ssl library by specifying a memory buffer that contains it instead of a path to a file containing the private key. (The private key will be read from a store (e.g. windows certificate store)).

Boost::asio::ssl doesn't provide this possiblity (the only method provided being to specify a path to the PKEY file) so I decided to extend the Asio::ssl with a custom context that provides this method.

How can I do that ?

I tried by extending the ssl classes with these:

The OpenSSL implementation extended:

class openssl_context_service_ext
    : public boost::asio::ssl::detail::openssl_context_service
{
public:
    typedef boost::uint8_t* pkey_buffer_t;

    openssl_context_service_ext(boost::asio::io_service& io_service)
        : boost::asio::ssl::detail::openssl_context_service(io_service)
    {
    }

 public:
    boost::system::error_code use_private_key(impl_type& impl,
        pkey_buffer_t pkey, boost::system::error_code& ec)
    {
... calls SSL_CTX_use_PrivateKey with the provided key buffer
    }
};

A new context service that uses this implementation:

class context_service_ext
      : public boost::asio::detail::service_base<context_service_ext>
{
private:
    // The type of the platform-specific implementation.
    typedef detail::openssl_context_service_ext service_impl_type;

public:
    typedef service_impl_type::impl_type impl_type;

    explicit context_service_ext(boost::asio::io_service& io_service)
        : boost::asio::detail::service_base<context_service_ext>(io_service),
          service_impl_(boost::asio::use_service<service_impl_type>(io_service))
    {
    }

    boost::system::error_code use_private_key(impl_type& impl,

        const detail::openssl_context_service_ext::pkey_buffer_t pkey,
        boost::system::error_code& ec)
    {
        return service_impl_.use_private_key(impl, pkey, ec);
    }


private:
    service_impl_type& service_impl_;
};

The problem is that when I try to create a basic_context with my new context_service the compiler complains (on good reason) that:
'use_private_key' : is not a member of 'boost::asio::ssl::basic_context<Service>'


To fix it I tried to extend the boost::asio::ssl::basic_context with the missing method, only that I don't have access to the private implementation (the context_service_ext) in the basic_context and there is no method provided to access it.

The basic_context extended:

template <typename Service>
class basic_context_ext
    : public basic_context<Service>
{
public:
    basic_context_ext(io_service& io_service, method m)
        : basic_context<Service>(io_service, m)
    {
    }

public:
    void use_private_key(const detail::openssl_context_service_ext::pkey_buffer_t pkey)
    {
        boost::system::error_code ec;
        service_.use_private_key(impl_, pkey, ec);
        boost::asio::detail::throw_error(ec);
    }
};

Is it even possible to achieve what I want ? Is there another way to add this new functionality ? Is there a reason why it wasn't provided in the current implementation of boost::asio::ssl  ?


Thank you!
vf
2012-11-22 22:22:10 UTC
Permalink
Have you tried using the capi engine in openssl?

On the client side it should be rather straight forward:

ENGINE_load_capi();
ENGINE* e = ENGINE_by_id("capi");
ENGINE_ctrl_cmd_string(e, "store_name", "MY", 0);
SSL_CTX_set_client_cert_engine(boost_ssl_context.native_handle(), e);

this will load the certificate the current user personal store and display the
use UI to select the certificate if there are more than one (openssl should
compiled in a certain way to enable Windows certificate UI).

On the server side it would a bit be more complicated.
I guess boost.asio does not provide this functionality since it bis not
portable. Once you have got the certificate in X509 form and the pk, loading
them should be trivial

X509* cert;
SSL_CTX_use_certificate(boost_ssl_context.native_handle(), cert);
EVP_PKEY* pkey = ENGINE_load_private_key(e, "my certificate", NULL, NULL);
SSL_CTX_use_PrivateKey((boost_ssl_context.native_handle(), pkey)
LN
2012-11-23 08:50:21 UTC
Permalink
Hi vf,

First of all thank you for your answer.

"Have you tried using the capi engine in openssl?"


I'm not familiar with CAPI and using it in the openssl. Some starting point/tutorial on how to do that would be great to have if you know a good one.


"On the server side it would a bit be more complicated.I guess boost.asio does not provide this functionality since it is not
portable. Once you have got the certificate in X509 form and the pk, loading
them should be trivial

X509* cert;
SSL_CTX_use_certificate(boost_ssl_context.native_handle(), cert);
EVP_PKEY* pkey = ENGINE_load_private_key(e, "my certificate", NULL, NULL);

SSL_CTX_use_PrivateKey((boost_ssl_context.native_handle(), pkey)"

I need this functionality on the server side.

I think I have achieved the same by extending the Boost providedboost::asio::ssl::context like this:

class ssl_context_ext : public boost::asio::ssl::context
{
public:
    ssl_context_ext(boost::asio::io_service& io_service, boost::asio::ssl::context_base::method m)
        : boost::asio::ssl::context(io_service, m)
    {
    }

    ~ssl_context_ext() {};

public:
    
    int use_private_key(boost::uint8_t* buffer)
    {
        int ret = 0;
        if (buffer != NULL)
        {
            ret = SSL_CTX_use_PrivateKey(impl(), (EVP_PKEY*)buffer); // I'm using an older version of boost and I don't have the native_handle() method
        }

        return ret;
    }
};

but I hoped to be possible to do the same by extending the implementation which is boost::asio::ssl::detail::openssl_context_service. In my opinion that would be the natural/C++ way because that is the class that contains the deatils of the openssl implementation.
With the code above I think I can use the Crypto API in windows to load the certificate and PKEY from the windows certificate store.


Anyway, I wonder why doesn't the basic_context have it's service_ member protected, not private as it is now... or at least have a public/protected method to get it from derived classes. Is it not supposed to be extended ?? because, the current implementation of boost::asio::ssl I think allows only adding new SSL implementation but it doesn't allow extending the existing interface. Am I missing something here ?

Also I wonder whether there is any platform independent implementation for dealing with certificates/PKEYs ? Or maybe a platform independent certificate store ? I couldn't find anything like this and that seems strange since certificates are extensively used for secure applications.

Thank you!


________________________________
From: vf <***@hotmail.com>
To: asio-***@lists.sourceforge.net
Sent: Friday, November 23, 2012 12:22 AM
Subject: Re: [asio-users] adding new functionality to the boost::asio::ssl context (new method for passing the private key)

Have you tried using the capi engine in openssl?

On the client side it should be rather straight forward:

ENGINE_load_capi();
ENGINE* e = ENGINE_by_id("capi");
ENGINE_ctrl_cmd_string(e, "store_name", "MY", 0);
SSL_CTX_set_client_cert_engine(boost_ssl_context.native_handle(), e);

this will load the certificate the current user personal store and display the
use UI to select the certificate if there are more than one (openssl should
compiled in a certain way to enable Windows certificate UI).

On the server side it would a bit be more complicated.
I guess boost.asio does not provide this functionality since it bis not
portable. Once you have got the certificate in X509 form and the pk, loading
them should be trivial

X509* cert;
SSL_CTX_use_certificate(boost_ssl_context.native_handle(), cert);
EVP_PKEY* pkey = ENGINE_load_private_key(e, "my certificate", NULL, NULL);
SSL_CTX_use_PrivateKey((boost_ssl_context.native_handle(), pkey)



------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
vf
2012-11-25 23:46:18 UTC
Permalink
Post by LN
I need this functionality on the server side.
I think I have achieved the same by extending the Boost provided
class ssl_context_ext : public boost::asio::ssl::context
{
    ssl_context_ext(boost::asio::io_service& io_service,
boost::asio::ssl::context_base::method m)
Post by LN
        : boost::asio::ssl::context(io_service, m)
    {
    }
    ~ssl_context_ext() {};
    
    int use_private_key(boost::uint8_t* buffer)
    {
        int ret = 0;
        if (buffer != NULL)
        {
            ret = SSL_CTX_use_PrivateKey(impl(), (EVP_PKEY*)buffer); // I'm
using an older version of boost and I don't have the native_handle()
Post by LN
method
        }
        return ret;
    }
};
yes, that will do but note generally it is advised to avoid passing 'raw'
pointers, as it is error prone.
Post by LN
Anyway, I wonder why doesn't the basic_context have it's service_ member
protected, not private as it is now... or at least have a public/protected
method to get it from derived classes. Is it not supposed to be extended ??
because, the current implementation of boost::asio::ssl I think allows only
adding new
Post by LN
SSL implementation but it doesn't allow extending the existing interface. Am
I missing something here ?
The service is templated in the context class declaration. It is so called
templated polymorphism as opposed to the usual virtual function based one.
Basically, you have to come up with a new service implementation which conforms
to the existing interface.
Consider the following member:

// Use an RSA private key from a file.
boost::system::error_code use_rsa_private_key_file(impl_type& impl,
const std::string& filename, context_base::file_format format,
boost::system::error_code& ec)


You could implement this function by interpreting the filename parameter as a
windoze certificate identifier, something like 'store name\certificate name' and
ignore the file format.
Post by LN
Also I wonder whether there is any platform independent implementation for
dealing with certificates/PKEYs ? Or maybe a platform independent certificate
store ? I couldn't find anything like this and that seems strange since
certificates are extensively used for secure applications.


Using openssl is as far as possible you can get to the portability. Loading
certifiates/pks from PEM files is the only portable thing. Everything else
(including the concept of stores) would be OS specific.

In the openssl source, you could have a look at the content of the e_capi.c.
Also, the new version of boost asio has an improved ssl implementation.
LN
2012-11-26 08:56:06 UTC
Permalink
Post by vf
The service is templated in the context class declaration. It is so called
templated polymorphism as opposed to the usual virtual function based one.
Basically, you have to come up with a new service implementation which conforms
to the existing interface.
There is no problem for me with that (templated polymorphism), but in my case I want to add a new method to the provided interface.
I want to do it not by changing the provided interface, but by extending it in a new basic_context_ext<Service> which inherits the base_context<Service>. 
This extended basic_context will have a new method for passing the PKEY as a buffer (notice that the Service implementation remains templated).
Also I will provide a new context_service that uses my own implementation of openssl_context_service (which only adds the new method "use_private_key" as a buffer).
I can't do it because in the basic_context_ext<Service> I cannot use the service_ member in the base class.
I still believe that should have been protected, not private, unless there is another good reason, that I'm missing, for which it is private.
Post by vf
// Use an RSA private key from a file.
 boost::system::error_code use_rsa_private_key_file(impl_type& impl,
     const std::string& filename, context_base::file_format format,
     boost::system::error_code& ec)
You could implement this function by interpreting the filename parameter as a
windoze certificate identifier, something like 'store name\certificate name' and
ignore the file format.
Yeah, but what if I want to use both methods for specifying the private key ? 
It doesn't look like the right solution but more like a workaround.
Post by vf
Using openssl is as far as possible you can get to the portability. Loading
certifiates/pks from PEM files is the only portable thing. Everything else
(including the concept of stores) would be OS specific.
I experimented a bit with OpenSSL engines (the CAPI one) and I could read the PKEY from windows store.
Thanks for pointing me to this.
Do you know what engines would be used for MAC, linux, etc ?
Post by vf
In the openssl source, you could have a look at the content of the e_capi.c. 
Also, the new version of boost asio has an improved ssl implementation. 
I'm forced to using boost 1.46 for now. By improved you mean providing more functionality, similar with what I need ?




________________________________
From: vf <***@hotmail.com>
To: asio-***@lists.sourceforge.net
Sent: Monday, November 26, 2012 1:46 AM
Subject: Re: [asio-users] adding new functionality to the boost::asio::ssl context (new method for passing the private key)
Post by vf
I need this functionality on the server side.
I think I have achieved the same by extending the Boost provided
class ssl_context_ext : public boost::asio::ssl::context
{
    ssl_context_ext(boost::asio::io_service& io_service,
boost::asio::ssl::context_base::method m)
Post by vf
        : boost::asio::ssl::context(io_service, m)
    {
    }
    ~ssl_context_ext() {};
    
    int use_private_key(boost::uint8_t* buffer)
    {
        int ret = 0;
        if (buffer != NULL)
        {
            ret = SSL_CTX_use_PrivateKey(impl(), (EVP_PKEY*)buffer); // I'm
using an older version of boost and I don't have the native_handle()
Post by vf
  method
        }
        return ret;
    }
};
yes, that will do but note generally it is advised to avoid passing 'raw'
pointers, as it is error prone.
Post by vf
Anyway, I wonder why doesn't the basic_context have it's service_ member
protected, not private as it is now... or at least have a public/protected
method to get it from derived classes. Is it not supposed to be extended ??
because, the current implementation of boost::asio::ssl I think allows only
adding new
Post by vf
  SSL implementation but it doesn't allow extending the existing interface. Am
I missing something here ?
The service is templated in the context class declaration. It is so called
templated polymorphism as opposed to the usual virtual function based one.
Basically, you have to come up with a new service implementation which conforms
to the existing interface.
Consider the following member:

// Use an RSA private key from a file.
  boost::system::error_code use_rsa_private_key_file(impl_type& impl,
      const std::string& filename, context_base::file_format format,
      boost::system::error_code& ec)


You could implement this function by interpreting the filename parameter as a
windoze certificate identifier, something like 'store name\certificate name' and
ignore the file format.
Post by vf
Also I wonder whether there is any platform independent implementation for
dealing with certificates/PKEYs ? Or maybe a platform independent certificate
store ? I couldn't find anything like this and that seems strange since
certificates are extensively used for secure applications.


Using openssl is as far as possible you can get to the portability. Loading
certifiates/pks from PEM files is the only portable thing. Everything else
(including the concept of stores) would be OS specific.

In the openssl source, you could have a look at the content of the e_capi.c.
Also, the new version of boost asio has an improved ssl implementation.




------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
vf
2012-11-26 22:48:25 UTC
Permalink
Do you know what engines would be used for MAC, linux, etc ?>
afaik all unix-like systems have a dedicated folders to keep the certificates
in, e.g /etc/ssl/certs. I am not aware of any engines for linux/mac, probably
just straight loading form files would do.
I'm forced to using boost 1.46 for now. By improved you mean providing more
functionality, similar with what I need ?
No new functionality, just fixing some annoyances mostly, see e.g.
http://beta.boost.org/users/history/version_1_48_0.html. (BTW, asio in boost
1.48 has a regression that, imo at least, renders it unusable, the next usable
version would be 1.49).
LN
2012-11-28 15:24:40 UTC
Permalink
Post by vf
Do you know what engines would be used for MAC, linux, etc ?>
afaik all unix-like systems have a dedicated folders to keep the certificates
in, e.g /etc/ssl/certs. I am not aware of any engines for linux/mac, probably
just straight loading form files would do.
I'm forced to using boost 1.46 for now. By improved you mean providing more
functionality, similar with what I need ?
No new functionality, just fixing some annoyances mostly, see e.g.
http://beta.boost.org/users/history/version_1_48_0.html. (BTW, asio in boost
1.48 has a regression that, imo at least, renders it unusable, the next usable
version would be 1.49).
I have tried to export the EVP_PKEY returned by ENGINE_load_private_key method
provided by OpenSSL CAPI engine to a PEM file by using PEM_write_bio_PrivateKey
and PEM_write_bio_RSAPrivateKey. The EVP_PKEY it is supposed to contain the
private key.

Writing to the PEM file worked, though the content of the file didn't seem
similar with that for a private key created with "openssl genrsa" (it was much
smaller).

I tried then to load the key from the PEM file and it didn't work.
PEM_read_bio_PrivateKey and PEM_read_bio_RSAPrivateKey fail to decode the
private key from the file.

I spent almost the entire day trying to understand what is happening until
finally I had the idea to use PEM_write_bio_PUBKEY to write the key and
PEM_read_bio_PUBKEY to read it back from the file.

To my surprise it worked ...

Now, the natural question is, does the OpenSSL CAPI engine method
ENGINE_load_private_key really return the private key or it returns the public
key ?

My knowledge is that in case of RSA the public and private key can be used
interchangeably and I have noticed that the code inside
ENGINE_load_private_key calls CryptExportKey with a PUBLICKEYBLOB for the BLOB
type. Does this mean that it actually exports the public key?

If it exports the private key, how can I write/read it to/from a PEM file
since the method described above failed ? (at least when loading back the key
from the file)

Thank you!
vf
2012-11-29 00:04:51 UTC
Permalink
Post by LN
I have tried to export the EVP_PKEY returned by ENGINE_load_private_key method
provided by OpenSSL CAPI engine to a PEM file by using PEM_write_bio_PrivateKey
and PEM_write_bio_RSAPrivateKey. The EVP_PKEY it is supposed to contain the
private key.
Writing to the PEM file worked, though the content of the file didn't seem
similar with that for a private key created with "openssl genrsa" (it was much
smaller).
I tried then to load the key from the PEM file and it didn't work.
PEM_read_bio_PrivateKey and PEM_read_bio_RSAPrivateKey fail to decode the
private key from the file.
I think it would be helpful if you show the code for exporting the private key,
and the error message/code produced by its loading.
Also the size of the privkey file would depend on the length if the key (1024
bits or whatever) and the method (are you sure it is RSA?).


I spent almost the entire day trying to understand what is happening until
Post by LN
finally I had the idea to use PEM_write_bio_PUBKEY to write the key and
PEM_read_bio_PUBKEY to read it back from the file.
To my surprise it worked ...
Now, the natural question is, does the OpenSSL CAPI engine method
ENGINE_load_private_key really return the private key or it returns the public
key ?
The capi engine loads the private key in capi_get_key() by using the
CryptGetUserKey(). The MSDN help for this API reads: "The CryptGetUserKey
function retrieves a handle of one of a user's two public/private key pairs.",
meaning either the exchange key pair and signature key pair, i.e.
AT_KEYEXCHANGE = 1 or AT_SIGNATURE=2. The capi engine determines which pair to
use by calling CertGetCertificateContextProperty(,CERT_KEY_PROV_INFO_PROP_ID,)),
the latter setting CRYPT_KEY_PROV_INFO.dwKeySpec to 1 or 2, and for all
practicals cases the keySpec should be 1.

I would not worry about the public key, the capi engine exports to a blob simply
for the purposes of converting the key to keep it the EVP_KEY structure.
In any way the public key business should be irrelevant here.

Loading...