Containers 101

(This is extracted from CIO Magazine by Paul Rubens. Clipped only for personal reference)

What are containers and why do you need them?

Containers are a solution to the problem of how to get software to run reliably when moved from one computing environment to another. Here’s what you need to know about this popular technology.

Docker exploded onto the scene in 2013, and it’s been causing excitement in IT circles ever since.

The application container technology provided by Docker promises to change the way that IT operations are carried out just as virtualization technology did a few years previously.

Here are answers to 13 of the most common questions related to this technology

  1. What are containers and why do you need them?
  2. How do containers solve this problem?
  3. What’s the difference between containers and virtualization?
  4. What other benefits do containers offer?
  5. What’s the difference between Docker and containers?
  6. Is there a standard container format?
  7. Why are all these companies involved in the Open Container Initiative?
  8. Are there any free open source container management systems?
  9. What commercial container management solutions exist today?
  10. How secure are containers?
  11. Which Linux distributions are suitable for use as a container host?
  12. What if you are a Windows shop?
  13. Will containers eventually replace full-blown server virtualization?

1. What are containers and why do you need them?

Containers are a solution to the problem of how to get software to run reliably when moved from one computing environment to another. This could be from a developer’s laptop to a test environment, from a staging environment into production, and perhaps from a physical machine in a data center to a virtual machine in a private or public cloud.

Problems arise when the supporting software environment is not identical, says Docker  creator Solomon Hykes. “You’re going to test using Python 2.7, and then it’s going to run on Python 3 in production and something weird will happen. Or you’ll rely on the behavior of a certain version of an SSL library and another one will be installed. You’ll run your tests on Debian and production is on Red Hat and all sorts of weird things happen.”

And it’s not just different software that can cause problems, he added. “The network topology might be different, or the security policies and storage might be different but the software has to run on it.”

2. How do containers solve this problem?

Put simply, a container consists of an entire runtime environment: an application, plus all its dependencies, libraries and other binaries, and configuration files needed to run it, bundled into one package. By containerizing the application platform and its dependencies, differences in OS distributions and underlying infrastructure are abstracted away.[B

3. What’s the difference between containers and virtualization?

With virtualization technology, the package that can be passed around is a virtual machine, and it includes an entire operating system as well as the application. A physical server running three virtual machines would have a hypervisor and three separate operating systems running on top of it.

By contrast a server running three containerized applications with Docker runs a single operating system, and each container shares the operating system kernel with the other containers. Shared parts of the operating system are read only, while each container has its own mount (i.e., a way to access the container) for writing. That means the containers are much more lightweight and use far fewer resources than virtual machines.

4. What other benefits do containers offer?

A container may be only tens of megabytes in size, whereas a virtual machine with its own entire operating system may be several gigabytes in size. Because of this, a single server can host far more containers than virtual machines.

Another major benefit is that virtual machines may take several minutes to boot up their operating systems and begin running the applications they host, while containerized applications can be started almost instantly. That means containers can be instantiated in a “just in time” fashion when they are needed and can disappear when they are no longer required, freeing up resources on their hosts.

A third benefit is that containerization allows for greater modularity. Rather than run an entire complex application inside a single container, the application can be split in to modules (such as the database, the application front end, and so on). This is the so-called microservices approach.  Applications built in this way are easier to manage because each module is relatively simple, and changes can be made to modules without having to rebuild the entire application. Because containers are so lightweight, individual modules (or microservices) can be instantiated only when they are needed and are available almost immediately.

5. What’s the difference between Docker and containers?

Docker has become synonymous with container technology because it has been the most successful at popularizing it. But container technology is not new; it has been built into Linux in the form of LXC for over 10 years, and similar operating system level virtualization has also been offered by FreeBSD jails, AIX Workload Partitions and Solaris Containers.

6. Is there a standard container format?

Back in 2015, a company called CoreOS produced its own App Container Image (ACI) specification that was different from Docker’s container specification, and at the time there was a risk that the newly-popular container movement would fragment with rival Linux container formats.

But later in the same year an initiative called the Open Container Project was announced, and later renamed as the Open Container Initiative (OCI). Run under the auspices of the Linux Foundation, the purpose of the OCI is to develop industry standards for a container format and container runtime software for all platforms. The starting point of the OCP standards was Docker technology, and Docker donated about 5 percent of its codebase to the project to get it off the ground.

The project’s sponsors include AWS, Google, IBM, HP, Microsoft, VMware, Red Hat, Oracle, Twitter, and HP as well as Docker and CoreOS

7. Why are all these companies involved in the Open Container Initiative?

The idea of the OCI is to ensure that the fundamental building blocks of container technology (such as the container format) are standardized so that everyone can take advantage of them.

That means that rather than spending resources developing competing container technologies, organizations can focus on developing the additional software needed to support the use of standardized containers in an enterprise or cloud environment. The type of software needed includes container orchestration and management systems and container security systems.

8. Are there any free open source container management systems?

Yes. Probably the best known and most widely used free and open source container management systems is Kubernetes, which is a software project that originated at Google. Kubernetes provides mechanisms for deploying, maintaining and scaling containerized applications

9. What commercial container management solutions exist today?

Docker Enterprise Edition is perhaps the best known commercial container management solution. It provides an integrated, tested and certified platform for apps running on enterprise Linux or Windows operating systems and cloud providers.

But there are many others, and several notable ones have a layer of proprietary software built around Kubernetes at the core. Examples of this type of management software product include:

  • CoreOS’s Tectonic pre-packages all of the open source components required to build a Google-style infrastructure and adds additional commercial features, such as a management console, corporate SSO integration, and Quay, an enterprise-ready container registry.
  • Red Hat’s Open Shift Container Platform  is an on-premises private platform as a service product, built around a core of application containers powered by Docker, with orchestration and management provided by Kubernetes, on a foundation of Red Hat Enterprise Linux.
  • Rancher Labs’ Rancher is a commercial open source solution designed makes it easy to deploy and manage containers in production on any infrastructure.

10. How secure are containers?

Many people believe that containers are less secure than virtual machines because if there’s a vulnerability in the container host kernel, it could provide a way into the containers that are sharing it. That’s also true with a hypervisor, but since a hypervisor provides far less functionality than a Linux kernel (which typically implements file systems, networking, application process controls and so on) it presents a much smaller attack surface.

But in the last couple of years a great deal of effort has been devoted to developing software to enhance the security of containers.

For example, Docker (and other container systems) now include a signing infrastructure allowing administrators to sign container images to prevent untrusted containers from being deployed.

However, it is not necessarily the case that a trusted, signed container is secure to run, because vulnerabilities may be discovered in some of the software in the container after it has been signed. For that reason, Docker and others offer container security scanning solutions that can notify administrators if any container images have vulnerabilities that could be exploited.

More specialized container security software has also been developed. For example, Twistlock offers software that profiles a container’s expected behavior and “whitelists” processes, networking activities (such as source and destination IP addresses and ports) and even certain storage practices so that any malicious or unexpected behavior can be flagged.

Another specialist container security company called Polyverse takes a different approach. It takes advantage of the fact that containers can be started in a fraction of a second to relaunch containerized applications in a known good state every few seconds to minimize the time that a hacker has to exploit an application running in a container.

11. Which Linux distributions are suitable for use as a container host?

Most Linux distributions are unnecessarily feature-heavy if their intended use is simply to act as a container host to run containers. For that reason, a number of Linux distributions have been designed specifically for running containers.

Some examples include:

  • Container Linux (formerly CoreOS Linux) — one of the first lightweight container operating systems built for containers
  • RancherOS — a simplified Linux distribution built from containers, specifically for running containers.
  • Photon OS — a minimal Linux container host, optimized to run on VMware platforms.
  • Project Atomic Host — Red Hat’s lightweight container OS has versions that are based on CentOS and Fedora, and there is also a downstream enterprise version in Red Hat Enterprise Linux.  
  • Ubuntu Core — the smallest Ubuntu version, Ubuntu Core is designed as a host operating system for IoT devices and large-scale cloud container deployments

12. What if you are a Windows shop?

In addition to running on any Linux distribution running version 3.10 (or later) of the Linux kernel, Docker also runs on Windows.

That’s because in 2016 Microsoft introduced the ability to run Windows containers in Windows Server 2016 and Windows 10. These are Docker containers designed for Windows, and they can be managed from any Docker client or from Microsoft’s PowerShell.

(Microsoft also introduced Hyper-V containers, which are Windows containers running in a Hyper-V virtual machine for added isolation.)

Windows containers can be deployed on a standard install of Windows Server 2016, the streamlined Server Core install, or the Nano Server install option which is specifically designed for running applications inside containers or virtual machines.

In addition to Linux and Windows, Docker also runs on popular cloud platforms including Amazon EC2, Google Compute Engine, Microsoft Azure and Rackspace.

13. Will containers eventually replace full-blown server virtualization?

That’s unlikely in the foreseeable future for a number of important reasons.

First, there is still a widely held view that virtual machines offer better security than containers because of the increased level of isolation that they provide.

Second, the management tools that are available to orchestrate large numbers of containers are also not yet as comprehensive as software for managing virtualized infrastructure, such as VMware’s vCenter or Microsoft’s System Center. Companies that have made significant investments in this type of software are unlikely to want to abandon their virtualized infrastructure without very good reason.

Perhaps more importantly, virtualization and containers are also coming to be seen as complementary technologies rather than competing ones. That’s because containers can be run in lightweight virtual machines to increase isolation and therefore security, and because hardware virtualization makes it easier to manage the hardware infrastructure (networks, servers and storage) that are needed to support containers.

VMware encourages customers who have invested in its virtual machine management infrastructure to run containers on its Photon OS container Linux distro inside lightweight virtual machines that can then be managed from vCenter. This is VMware’s “container in a VM” strategy.

But VMware has also introduced what it calls vSphere Integrated Containers (VICs). These containers can be deployed directly to a standalone ESXi host or deployed to vCenter Server as if they were virtual machines. This is VMware’s “container as a VM” strategy.

Both approaches have their benefits, but what’s important is that rather than replacing virtual machines, it can often be useful to be able to use containers within a virtualized infrastructure.

Blockchain for the Supply Chain

A question of trust in trade

The modern supply chain is long on data and short on trust. Historically, mistrust between organizations, including fear that information might be passed on to a competitor has stopped organizations from sharing data. In turn, even when information is shared, it’s often not fully trusted.

The Blockchain

Blockchains – distributed ledgers that create a permanent and shared record of every transaction associated with an asset – create an unbroken chain of trust. Each record is time-stamped and appended to the event before it. Blockchains have the potential to generate breakthroughs in three areas: visibility, optimization and demand.

Data records on the blockchain can be accessed only by authorized participants; this can be all participants or only those that need a particular portion of the data. Data ownership and access can be anonymous yet securely identified between partners who require verification. In short, it can be widely shared and protected at the same time.

Is Blockchain the right answer?

How can you create value, collaborate and compete with blockchains? Organizations looking for new ways to create value, collaborate and compete must consider the following questions:

  1. Which of your existing partners and what other institutions need to participate in the blockchain to make it worthwhile?
  2. How much and which data should you reveal and to whom?
  3. What data do you have that might have value to others and is there a way for you to benefit from that value?
  4. Which intermediaries in your ecosystem exist simply to certify trust or handle complexity?
  5. How would new access to customer – or supply chain partners’ – demand signals change my operations?

The quirky Lakh

One of the more interesting nuances of currency conversion in Myanmar (and more broadly in India) is the concept of the Lakh.

So the “Lakh” is a numerical value of 100,000 but is actually written as 1,00,000 according to the digit grouping convention.

The usage of the Lakh (or Lac) is surprisingly commonplace if you visit any South Asian country which includes Myanmar, India, Nepal, Pakistan, Bangladesh, and Bhutan).  So when discussing large-ish amounts of money, I was initially confounded trying to make conversions in my head when people would mention “10 Lakhs” instead of “1 Million” or “half a Lakh” instead of “50 Thousand”.  To the point that I considered creating a new calculator app or site that convert Lakhs to “metric” values.

But there you go.  Might come in handy the next time you do business in South Asia.

 

Cryptocurrencies

An interesting offshoot of blockchain technologies, and one of the key drivers and catalysts for it’s adoption and popularity – is the crypto currency.   As of writing, the booming crypto market has just had a recent correction but still holding ground. Nothing like the predicted bloodbath, but that’s not to say it one isn’t around the corner.

To quench my curiosity I’ve gone in bought some Ethereum, Ripple, and NEO to spread the portfolio.   I have stayed away from Bitcoin for the time being, as it just feels clunky and over valued – not that there’s any basis for value to be attributed to any crypto currency.   However, I have chosen the three currencies for the following reasons:

  • Ethereum – in my view is primarily a blockchain platform which doubles as a cryptocurrency in order incentive miners to process the chain.   It’s backed a significant consortium of industry heavy weights which lends credence to the platform.
  • Ripple – is unique in that it’s a closed system controlled by Ripple that facilitates transactions between financial institutions.  In many ways it represents the antithesis of blockchain’s purpose of being de-centralised.  However it has the backing of large financial institutions.
  • NEO – is China’s largest open source blockchain initiative.   Given most of the mining operations take place in China, it seems to have a good basis to succeed.  At the same tine, it also poses some potential risk if heavy Chinese government regulation comes into play.

 

Going Agile

When clients talk about delivering in an Agile approach, it’s often a case of assessing where to place their appetite and organisational maturity along an ‘Agile spectrum’.  More often that not it typically ends up being an interative waterfall approach.  But here’s my Agile diagram that’s done its fair share of rounds in our organisation.

agile

To me, Agile is less about methodology and more about organisational culture in how they think about and approach delivery.

The Venture Capitalist

Over the weekend I met with some VC’s from Silicon Valley and Switzerland, which prompted me to take a crash course in the world of venture capitalists and startups.

What is Venture Capital?

There are two types of financing: debt and equity. Debt financing means: I’ll give you this money, if you promise to pay it back to me with interest. Equity financing means: I’ll give you this money in exchange for owning part of your company. Venture capital (VC) is financial capital provided to early-stage, high-potential, companies in exchange for equity in the companies it invests in.

What is the structure of a venture capital firm?

A VC firm is comprised of limited partners (LP’s) and general partners(GP’s). Limited partners (pension funds, educational endowments, foundations, insurance companies, wealthy people) are investors in the venture capital firm’s fund. General partners (ex-CEO’s and founders) are are the ones who run the fund, source deals, make investment decisions, and maintain the portfolio. Fun Fact: 99% of the fund comes from LP’s while 1% comes from the GP’s.

Industry norm is that 20% of deal profit goes to GPs while LPs get the remaining 80%. Additionally, 2% of the fund size goes toward fund expenses, such as paying the salaries of the GPs and any additional staff (associates, analysts, etc). That means, if you have a $100M fund, $2M of that money automatically goes to the GPs who decide how to use it.

What are the different sources of capital?

Angels: A wealthy individual who invests their own money, usually in a business in which they have expertise. These are ex-founders and wealthy people.

Super Angels: An angel who is the lead investor of a group of angels (syndicates).

Micro Seed Funds/Accelerators: Small funds that do a lot of small investments in exchange for a small amount of equity. Accelerators tend to sell themselves as providing additional support such as mentorship, business connections, and a range of professional services (such as help on the legal part of your company). Examples of these are Y-CombinatorKima VenturesTechstars, etc.

Venture Capital: Medium funds that invest greater amounts of capital for greater amounts of equity. Examples of these are Andreeseen HorowitzSequoia CapitalGoogle Ventures, etc.

Growth Equity: Big funds that invest huge amounts of money to expand a successful business model. One example is Summit Partners.

What are the different types of rounds?

  1. Seed: Initial funding to build initial product and prove business model
  2. Series A: Build core team and launch core product
  3. Series B: Expand team and expand product portfolio
  4. Series C: Scaling the business model
  5. Series D+: Geographic expansion of business

If you are wondering what round a company is in or are curious about its funding history, take a peek on crunchbase.com.

How does venture capital work?

  1. Entrepreneur gets introduction to multiple VC firms
  2. Entrepreneur pitches business to VC firms
  3. Term sheet written if VCs want to invest
  4. Build business further
  5. VCs repaid through: acquisition, IPO, or bankruptcy

Gartner top 10 Technology Trends 2018

The basis of Gartner’s trends depends on whether or not they have the potential to disrupt the industry, and break out into something more impactful.

gartner-10-tech-trends-2018

 

The top 10 strategic technology trends, according to Gartner, are:

Intelligent

  1. AI foundation: Last year, the organization included artificial intelligence and machine learning as its own trend on the list, but with AI and machine learning becoming more advance, Gartner is looking at how the technology will be integrated over the next five years. “AI techniques are evolving rapidly and organizations will need to invest significantly in skills, processes and tools to successfully exploit these techniques and build AI-enhanced systems,” said David Cearley, vice president and Gartner Fellow. “Investment areas can include data preparation, integration, algorithm and training methodology selection, and model creation. Multiple constituencies including data scientists, developers and business process owners will need to work together.”
  2. Intelligent apps and analytics: Continuing with its AI and machine learning theme, Gartner predicts new intelligent solutions that change the way people interact with systems, and transform the way they work.
  3. Intelligent things: Last in the AI technology trend area is intelligent things. According to Gartner, these go beyond rigid programming models and exploit AI to provide more advanced behaviors and interactions between people and their environment. Such solutions include: autonomous vehicles, robots and drones as well as the extension of existing Internet of Things solutions.

Digital

  1. Digital twin: A digital twin is a digital representation of real-world entities or systems, Gartner explains. “Over time, digital representations of virtually every aspect of our world will be connected dynamically with their real-world counterpart and with one another and infused with AI-based capabilities to enable advanced simulation, operation and analysis,” said Cearley. “City planners, digital marketers, healthcare professionals and industrial planners will all benefit from this long-term shift to the integrated digital twin world.”
  2. Cloud to the edge: Internet in the Internet of Things has brought up the notion of edge computing. According to Gartner, Edge computing is a form of computing topology that processes, collects and delivers information closer to its source. “When used as complementary concepts, cloud can be the style of computing used to create a service-oriented model and a centralized control and coordination structure with edge being used as a delivery style allowing for disconnected or distributed process execution of aspects of the cloud service,” said Cearley.
  3. Conversational platforms: Conversational platforms such as chatbots are transforming how humans interact with the emerging digital world. This new platform will be in the form of question and command experiences where a user asks a question and the platform is there able to respond.
  4. Immersive experience: In addition to conversational platforms, experiences such as virtual, augmented and mixed reality will also change how humans interact and perceive the world. Outside of video games and videos, businesses can use immersive experience to create real-life scenarios and apply it to design, training and visualization processes, according to Gartner.

Mesh

  1. Blockchain: Once again, blockchains makes the list for its evolution into a digital transformation platform. In addition to the financial services industry, Gartner sees blockchains being used in a number of different apps such as government, healthcare, manufacturing, media distribution, identity verification, title registry, and supply chain.
  2. Event driven: New to this year’s list is the idea that the business is always looking for new digital business opportunities. “A key distinction of a digital business is that it’s event-centric, which means it’s always sensing, always ready and always learning,” saidYefim Natis, vice president, distinguished analyst and Gartner Fellow. “That’s why application leaders guiding a digital transformation initiative must make ‘event thinking’ the technical, organizational and cultural foundation of their strategy.”
  3. Continuous adaptive risk and trust: Lastly, the organization sees digital business initiatives adopting a continuous adaptive risk and trust assessment (CARTA) model as security becomes more important in a digital world. CARTA enables businesses to provide real-time, risk and trust-based decision making, according to Gartner.

Swiftpoint GT Mouse

I’ve actually had this mouse for over a year and realised it deserves its own post as my favourite pointing device (aka mouse).   The Swiftpoint GT has actually been around since 2014 after a successful Kickstarter campaign, and currently retails for about $149 USD ($190 SGD).

swiftpoint gt

Features

  • Extremely portable, with a tiny form factor that makes it possible to use on the palm rest of your laptop – super handy on flights.  It comes with a handy palm-rest mat to keep it secure on your latop.  And almost forgot to mention it weights only 24g.
  • USB receiver. Doubles as a charger for the GT which has a rechargable internal battery.  The GT also connects via Wireless and Bluetooth 4, although I end up using the dongle for quick and reliable connection.
  • Magnetic parking spot.  This is basically a circular rubber mat with a magnet which you can stick on the outside of your laptop, which you can then attach the GT to carry it around.  Works well, but easy to bump off if you knock it against something.
  • Gesture-based scrolling and panning.  The unique feature about the GT is an extra nib at it’s base which allows you to swipe the mouse from side to side to pan or flick between pages, or drag the whole unit up and down for scrolling.

swiftpoint-gt-modes

Things to watch

  • It definitely takes a little getting used to.   Most of my friends who try and use the GT find it extremely frustrating.  However, after using it for a few days, I find it not only comfortable and ergonomic but extremely productive.  The GT has a patented ergonomic design where you hold the mouse with a pen-like grip that feels natural to use.  Even though I initially bought it for travel and cafe visits, I’m now using it as my main desktop mouse.

The-Swiftpoint-is-a-miracle-in-engineering-and-design

  • Expensive.   For a price tag of $149 USD it’s definitely on the pricier end for any mouse.   But would I buy it again? I’d say yes.   It’s lasted a full year without any problems (I’ve gone through 3 desktop mice in the same period).   I’ve just gotten used to ergonomics of the design where I feel a tang

 

Reach the sick [Snippet]

According to the World Health Organisation, 400 million people do not have essential health services.  But emerging technologies can enable us to deliver health care worldwide in ways previously impossible.  For example, during the 2014 Ebola outbreak in Africa, mobile-tracking technology was used to predict the spread of infection.  Text messages and electronic mapping have also tracked supply levels of malaria medications in remote locations and, in so doing, ensured that medecines are available where they are needed.  These are peiple in areas where infrastructure is limited and health care professionals are scarce-being reached in real time.

– Joe Jiminez, CEO Novartis

Security Framework

A wanted to define an overarching security framework to encapsulate all the core elements that needed to be considered as part of ‘Security’.   This forms a starting point to delve into specific areas which I’ve bulleted to delve deeper.

security

Identity & Access Management

  • Authentication
  • Authorisation
  • User groups, role, and permission management
  • Directory Services
  • Single Sign On (SSO) and Federation

Network & Infrastructure

  • DMZ, firewalls, and reverse Proxies
  • Intrusion and virus detection
  • Server hardening
  • Physical security

Compliance

  • Security policies (PCI, PII, privacy, cookie laws,)
  • ISO27001
  • Security Review and Risk Assessment
  • Penetration Testing
  • Audit & Logging
  • Monitoring & reporting

Data & Information 

  • Encryption & hashing
  • Transmission
  • Storage
  • Protocols
  • Archiving