Over the last two blogs, I have explained the framework for public cloud discovery use cases, and introduced some of the considerations for effective cloud discovery. In this post, I will review some of the actual methods & technologies for leveraging cloud discovery.
- Cloud Providers API - Natural, all the cloud providers allow API access to manage the cloud. In fact, API is the only valid way for cloud management (as also the standard consoles to cloud providers are a wrapper for the Cloud API). For example, you can use this AWS API to get all the information on ec2 nodes. For an even more effective discovery, several cloud providers have specific API for management of configuration. For example, AWS offers the AWS Config which provides a way to keep track of the configurations of all the AWS resources associated with your AWS account. Google offers similar API to manage Google cloud assets. One important aspect to remember that for leveraging the specific asset APIs, you should sure they are enabled and configured correctly.
- Agent-less/Agent base discovery - While the usage of API is essential to capture configurations, you can only get so far with it. The resources that are tracked by the cloud providers do not include all the resources that you need to track for a complete configuration management. As you recall, the model of public cloud is of shared responsibility. As such, you will want to ensure that also resources that are managed by you, like the software that is installed, specific configurations and patches are being captured as well. In order to achieve it, a deeper method of discovery needs to be applied. Universal Discovery can be used to apply ‘standard discovery’ that can either be agentless (hence running remote commands on the specific cloud host instances) or databases, for example. Universal discovery can also deploy agents on the specific nodes that are deployed in the cloud, and capture much most information than the standard config APIs will reveal. Specific vendors like AWS also provide more capabilities to discover these resources, like the AWS System Manager. Leveraging the System Manager agents, you can get more information that is directly derived from your ec2 instances. Moreover, the information from system manager can also be consumed via the AWS config. Universal discovery can integrate with AWS system manager (as well of-course to leverage its own agent or agent less discovery capabilities) to capture the required information.
Here is an example of sample discovery results from AWS Discovery:
- Scheduled vs. event base - Another important aspect to take into account when planning the discovery of resources deployed on the cloud, is whether to leverage only scheduled activities, or also to apply event base discovery. Leveraging only the scheduled discovery, means that your resources will be updated just at specific intervals, which obviously will have a gap than the constant change of configurations in the cloud. Use this method only if you plan to leverage the discovery just for the more steady resources, or you leverage the discovery results for activities that are less time sensitive (specific aspects of software asset management can leverage this capability). Sources for events can differ from one cloud provider to another. In AWS the usage of the AWS config which can generate an event and activate a lambda function, is usually the best way to achieve constant update of data.
- Tag management - Another important aspect of cloud discovery is the ability to capture specific items that are commonly used in cloud deployment. Tags are the best example for this. Tags help you manage your instances, images, and other cloud resources, and you can use them to assign your own metadata to each resource. Having a consistent sets of tags for your resources (from example to specific application names, owners, versions etc), can help you to have better control of your cloud resources. Having the ability to collect and classify the information that is stored in these tags is important of-course to ensure you can actually use the tags effectively.
There are many other data resources that can be a very good source of data that can be collected and analyzed. For example, VPC flow logs in AWS that are usually used for troubleshooting of network issues, or for security purposes, can also be a great tool to map and analyze the interdependence of resources in the cloud. Other tools can be auditing tools (like cloud trail that can help you retain AWS account activity related to actions across your AWS infrastructure) or monitoring tools like Cloud watch that is focused on performance metrics.
The cloud has changed the way that IT and developers can create and deploy new services for their clients, and enabled a much faster time to market with mass scalability and automation. Leveraging the right tools, including ability to discover and manage all these resources, can ensure that on top of the cloud benefits you can also keep full visibility into all your accounts across all your cloud providers.
Read the other blogs in this series:
Hybrid Cloud Management