certfetcher - The easy way to grab public key certificates from available HTTPS, LDAPS, and other


If you've been in IT for most than ten minutes, and if you are working with any sort of encryption, and if you are security-conscious at all, it is likely that you have dealt with certificates and needed to get a service's certificate chain sooner than later.  This is usually accomplished by going into some tool and choosing one of a few objects in the certificate chain, and being sure to choose the Public Key, and being sure to uncheck the Private Key checkbox, and finally exporting the file, and then realizing it's in the wrong format after you logout, and starting all over.  After this is done you still do not know if what you have exported is correct for a client to use for a connection, so you import it using odd commands into a Java truststore (maybe called a keystore) or some other application's list of trusted certificates, and then it hasn't worked, and you've lost more hairs than you can count.

Understanding exactly how PKI works makes this entire process a little silly; while able to be performed using graphical tools, it is one of those processes that is much easier using the command line.  Of course, the openssl commands to pull it off are maybe a little daunting, kind of like using gcc to compile for the first time, or the 'rpm' command to query for a package's list of files, but with a little bit of experience the command line is much faster, and provides a way to test the data's validity directly against the service where it matters.  Of course, while you've been in IT for more than ten minutes, you aren't socially broken enough to bother figuring out those commands that will make this a one-second job instead of a ten minute clicking session; you have a life, and limited memory in your brain that makes it hard enough to keep your kids' names straight, much less cryptic parameters for commands you use once per year or so.   Luckily for you, I do not have a life, and have too much time on my hands, so here is a tool that will help rid you of your woes.  Because I have no creativity to go along with my lack of a social life, I have named this tool, "certfetcher" because, well, it fetches certs.

Why does understanding PKI make the graphical process silly and the command line process easier, you might ask?  Well that's because PKI has nothing to do with GUIs, and has everything to do with the connection that the SSL/TLS client will eventually make to the SSL/TLS server, and the data which will go back and forth.  Here's a little analogy for you.

Bob, who has a social life greater than my own, walks around downtown and sees Sally, a true beauty whose name is currently not known to him.  He wants to find out her name so that he can talk to her, so of course..... he goes to some social media site (Google seems likely) and quickly takes a picture of Sally with his phone.  With a little bit of apprehension he posts the picture, where he saw her, and desperately hopes that his "friends" will give him her name so that he can talk to her.  Eventually somebody answers, he gets the name, he walks up and says, "Hi Sally" which she finds only slightly creepy since she does not yet know his name, but plays along so that she does not hurt his feelings since they're obviously acquaintances.  It took a lot of work in a roundabout way, but finally Bob managed to get Sally's name, and then they could communicate.  Hooray!

Unfortunately, in the time Bob took to find out Sally's name, George walked up and said, "Hi, I'm George; what's your name?" to which Sally replied, "Sally" and so while Sally and Bob are talking, Sally already has a date with George.  Poor Bob... he should have just tried introducing himself rather than trying the silly, roundabout way of getting the necessary information.

This ridiculous (yes, I know it is too... I just came up with it and it's late at night) example is a little valid because it's clear to us that finding out Sally's name was just a matter of asking.  Sure it's possible to get the necessary information another way, but Sally is willing to just tell you if you ask.  Go ahead... ask her.

PKI is similar with certificates for various services.  In order to import a root or intermediate certificate into a truststore (or equivalent) the certificates can be retrieve in a roundabout way, or can be requested directly from the service that will eventually be contacted.  Sure, you should validate that the cert you find via DNS or IP or whatever is valid, but that's a step you should take anyway, and never do anyway, and it's something that Google will tell you with about one query, so that step, if you really take it (don't lie), is nothing compared to getting the public certificate in the first place.  The method used to go through various tools to export a CA or intermediate certificate is a lot of effort, and usually unnecessary since services usually advertise their certificate chain, or at least one part of the chain above the main certificate, for you.  How can you see it?  Well, for that we get into those scary openssl commands.

Tangent: You should learn openssl commands.  They are great for troubleshooting SSL services; at the OSI model layers where netcat only provides up through layer four, openssl can provide verification of functionality up through layer five and even let you do things up at layer seven if you have some idea of how that top layer works (for example this is trivial with HTTPS and several mail protocols).  Not learning openssl commands will not necessarily mean the end of your relevance any more than not learning the command line necessarily means your skills are irrelevant, but it does mean that you are likely missing out on better, faster, more-complete ways of doing things.  Examples are prevalent online and cover all kinds use cases; sure, it's new terminology (PKI in general is) and a bit of trial and error, but it's a skill that you will relish when you can show in about ten seconds, conclusively, for free, whether or not that new HTTPS service is working while on vacation using a slow connection, while all of the guys back at the office couldn't agree one way or another after hours of checking with web browsers and web-based "check-your-website" services, for which they paid.

Back on track, PKI works using public and private keys, and the public key is linked to a certificate which is basically a bunch of information about the associated public key which is then signed by an authority, usually controlled by a third-party.  For example, your web server may have a wildcard certificate with a single subject line (and no alternative subjects) of cn=*.yourdomain.com which means you can use it for any sites that match *.yourdomain.com (but not yourdomain.com itself, or things beneath *.yourdomain.com).  That public key will be signed by (for example) Digicert, a CA that is built into all kinds of SSL clients, like common web browsers (Firefox, Chromium, Opera, etc.), because it is trusted by those clients/browsers.  When your customers (using those web browsers) connect to https://www.yourdomain.com/ the first thing they do after the three-way TCP handshake is start an SSL handshake with the server.  At this state your server gives the client the proper certificate (which you imported) and the associated Public Key, all signed by Digicert.  Without that key being signed by a trusted third party (Digicert as the CA), the browser would have no idea if the certificate given to it was valid for that domain or if it was somehow being redirected to my site, which is hijacking your traffic and making your users feel safe even though I am stealing their passwords.

Because the third-party CA is trusted by the web browser (which has a copy of its Public Key stored in its truststore) and because the certificate with a subject of 'cn=*.yourdomain.com' is signed by the CA, the website certificate is also trusted.  Trust is a big deal in PKI and is the whole reason that the PKI industry, and really any other industry requiring security including everything around eCommerce, works online in its current form.  My site, the malicious one designed to steal passwords, could create a new certificate with the same subject name as yours, but I should never be able to do that with a trusted CA backing me since I am not the legitimate owner of the yourdomain.com domain name, so any set of keys that I use to fake a real site will be seen by the browser as a fake set of keys and result in a big warning displayed to the end user advising them to run away quickly.

On the technical side when the client (web browser in this case) connects to the server and starts the SSL Handshake, the server sends certificate information back to the client.  Because this includes the CA that signed the HTTP server's SSL certificate, all that is needed to tell a client to trust that CA is to store it in a place where trusted certificates are stored, and thus we have a truststore.  'Truststore' is a term often used in Java applications, but the concept is basically a place where trusted certs are stored, whether those be from third-party CAs (as shipped by web browsers or other SSL-enabled utilities) or your own self-signed certificates generated automatically by eDirectory or SLES when you first build an environment.

Why does this matter, really?  What is wrong with clicking "Accept this untrusted certificate permanently?" once in a while?  If it is not clear to you, that's no problem, since it is not clear to most.  Because it is unclear you have likely noticed that web browsers have become significantly more-obnoxious about untrusted certs in recent years.  It used to be little more than accidentally bumping the [Enter] key when the page loads, and now big red screens block out the target webpage and default to going to a homepage or the previous page or something else "safe" with wording like,"Get me out of here!"  Regular end users don't know about PKI, but you're an administrator so these details matter.

The reason your browser does not trust what is called a "self-signed" certificate (a certificate with no CA), or CAs that were not shipped with the browser (my CA for my malicious website, for example), is that there is no reason to trust them.  If a CA cannot demonstrate that it does a good job of verifying its customers are who they claim to be, then it is not accepted by vendors like Mozilla, Google, or Opera into browsers like Firefox, Chrome, or Opera.  The CAs that are accepted have theoretically done a good job verifying that they know what they are doing, and also ensure that their customers prove, somehow, that they are who they claim to be (not me posing as yourdomain.com's owner).  When this process breaks down CAs go out of business, and appropriately so since their entire value addition drops to zero and their company value, accordingly, goes with it.  There are examples of this happening, and usually the next day all browsers get emergency updates with a list of new CAs that no longer include the now-defunct CA.  If interested, Google for 'diginotar', a Dutch CA that failed to do things properly and was quickly out of business.

What does all of this have to do with a tool?  Well as I mentioned the SSL Handshake includes the CA certificate that signed the server's SSL certificate.  That CA certificate is compared against those in the trust store and, if present, the server's SSL certificate is verified to have been signed by the CA.  Other checks are also performed, such as verifying the subject name matches the connection information used to access the service, the certificate used is not being used too soon or too late (expiration), and that it is being used for a valid purpose (some certificates are only allowed for encryption, while others are only allowed for signing, while others have other purposes).  If all of the checks, well, check out, then the application is allowed to communicate with the server and things flow nicely.  If not, errors are returned with big red screens.

Stepping away from the end user world for a second, this tool is not necessarily designed to export certificates for web browsers.  The purpose here is to allow administrators and other technical folks to easily get a CA or intermediate certificate directly from a service into a file or other truststore.  This is often the case when one service connects to another, for example when a web application connects to an LDAP directory for authentication purposes, or a web application connects to a JDBC-based database backend for relational information.  SSL connections happen all over and in IT, SSL is used to protect communication that will never be visible to an end user, or any user, and thus will never have the option of clicking "Proceed anyway" despite bright red warning screens.  SSL is also used in cases where a CA may not be a trusted third-party CA, but is instead an internal, organization-maintained CA.  It may not be trusted by the whole world, but it may still be trustworthy enough to use internally based on the experience and technical prowess of the administrators within the organization.  In this case going to Google and requesting a certificate may not be an option.  Certificates may also be used in places where web browsers are not valid and cannot connect, either because the certs are used for non-HTTP services, or because they are only visible via protected, firewalled sockets from one machine to another (neither with a "proper" web browser).  The certificates used still need to be trusted, and there are still other ways to get them perhaps, but using a tool that simulates what the client and server will eventually do not only gets the data, but verifies other pieces are setup correctly, such as those firewalls, VLANs, or other networking pieces that can get in the way.

To use certfetcher currently start simply by decompressing the certfetcher.bz2 file into a 'bin' directory of your choice which is part of your desired user's PATH and then set as executable:

bunzip2 /path/to/certfetcher.bz2 > ~/bin/certfetcher

chmod x ~/bin/certfetcher

With that completed, simply run it specifying the desired IP address and port for the SSL connection.

certfetcher 443

Note that, of course, the system running this script must be able to fully establish the SSL connection on the specified machine; if not , copy the script to that machine and run it from there directly which should usually work.  Note that this does not, yet, support IPv6 addresses.  Also note that files are created in the local directory with the public certificate information in PEM/B64 format for use by anybody with rights; this is not a security risk.



How To-Best Practice
Comment List