2023/08/20

Secure Development Operations Week 3

Infotech

Secure Development Operations Week 3

Containers

1. What is software containerisation?:
   
   
   
   • Software containerisation is an OS virtualisation method that is used to deploy and run containers without using a virtual machine (VM).
   • Containers are isolated from each other, and multiple containers can run on the same hardware. This
     configuration helps us use hardware more efficiently and can help improve our application’s security.
   • Containers can run on physical hardware, in the cloud, VMs, and across multiple OSs.
2. Containers:
   
   
   Containers are a technology used in computing to isolate applications and allow them to run in a controlled environment. They differ from Virtual Machines (VMs), which also isolate applications but do so in a more resource-intensive way.
   
   The reason VMs are resource-intensive is that they provide full process isolation for applications by virtualizing the hardware. This means that each VM includes a full guest operating system, which consumes substantial computational power. Essentially, each VM is like a separate computer running inside your actual computer, complete with its operating system and all the overhead that comes with it.
   
   Containers, on the other hand, take a different, more efficient approach. Rather than virtualizing the hardware, they leverage the low-level mechanics of the host operating system. In doing so, containers provide most of the isolation that VMs offer but at a fraction of the computing power. Unlike VMs, containers share the host’s operating system kernel and run as isolated processes within the user space of that OS.
   
   The result of this approach is that containers are much lighter and faster than VMs. They can handle more applications, require fewer VMs, and take up less space, with container images typically being tens of megabytes in size. This efficiency allows for the quick and reliable execution of software packages, including the code and all dependencies needed to run applications, in any computing environment.
   
   In summary, containers are a modern solution for packaging and running applications in a way that provides isolation like VMs but does so more efficiently and with less overhead. By utilizing the host operating system’s mechanics, containers offer a streamlined and resource-saving way to deploy and manage applications across various environments.

Introduction to Docker

1. Docker: Docker is a tool that revolutionizes the way developers and system administrators deploy applications. It builds on the concept of containerization, allowing software to be packaged with all its dependencies into isolated environments called containers, ensuring consistency across different stages of development, testing, and production.
   
   Let’s break down what Docker does and why it’s beneficial:
   • Docker and Containers: Containers are isolated environments where applications run with all the resources they need, such as libraries and other dependencies. This isolation ensures that the application behaves the same way, regardless of where it’s running. Docker makes creating and managing these containers easy and efficient.
   • Portability: Docker packages applications into a “standardized unit” for development. This means that a container holding an application and its dependencies can run anywhere Docker is installed, whether it’s a developer’s laptop, a testing server, or a cloud provider like Azure. This portability eliminates the common problem where code behaves differently on different machines.
   • No Need for a Hypervisor: Unlike Virtual Machines, Docker doesn’t use a hypervisor. This makes Docker more efficient, as it communicates directly with the host operating system, and the containers share the host’s OS kernel. It means you can run Docker on your desktop or laptop for development and testing without requiring extensive system resources.
   • An Open-Source Project and a Company: Docker is not only the name of the technology but also a company that actively promotes and evolves it. It’s an open-source project, meaning developers around the world contribute to its improvement. Collaboration with various vendors, including cloud providers and operating system manufacturers like Microsoft, ensures wide compatibility and support.
   • Time-Saving and Simplification: Docker saves time because there’s no need to configure hardware or install operating systems and software to host a deployment. Once a Docker container is configured, it can be deployed repeatedly without further adjustments.
2. How to deal with different OS’s issues in Docker: Docker containers do utilize the host operating system’s kernel, and this can lead to some differences in behavior if the host OS is different. However, Docker is designed to minimize these differences by providing a consistent runtime environment for the application.
   • Containerization: Docker containers encapsulate not just the application but also all its dependencies, libraries, and binaries needed to run the application. This ensures that the application has everything it needs to run, regardless of the underlying host OS.
   • Docker Images: The application and its dependencies are packaged into a Docker image. This image includes a lightweight user-space “OS” that the application expects. This user-space can be different from the host OS and ensures that the application sees the same “system” regardless of where it’s run.
   • OS Kernel Compatibility: Docker is primarily designed for Linux, and different Linux distributions tend to have compatible kernels. If you’re running Docker on a non-Linux OS like Windows or macOS, Docker provides a Linux VM or uses a compatibility layer to execute the containers. This way, the containers still “think” they are running on a Linux kernel.
   • Careful Configuration: Developers and system administrators must still be mindful of potential differences and test their Docker containers across different host OSs if they suspect that the host OS might affect behavior. There may still be differences in performance or behavior tied to specific kernel features or hardware interactions.
3. Docker’s architecture:
   
   
   Docker’s architecture is designed to facilitate the development, deployment, and running of containerized applications. It operates as a client-server system, where both parts can run on the same host or on different machines. Here’s a breakdown of how it works:
   
   Docker Client and Server:
   • Docker Client: This is the command-line tool that developers or administrators use to interact with Docker. When you run a command like docker run, the client sends that request to the Docker daemon, which is the server part of the system.
   • Docker Server (Daemon): The daemon is responsible for all the heavy lifting associated with building, running, and managing containers. It listens for requests from the Docker client, either locally or remotely, and manages all aspects of the container lifecycle.

   The communication between client and daemon is done through a REST API, allowing flexibility and the possibility to have the client and server running on different hosts.
   
   Images and Containers:

   • Images: Docker uses images as the blueprint for containers. An image contains all the necessary files, dependencies, and information needed to create and run a container. It’s a snapshot of a file system that can be used to create containers.
   • Containers: Containers are running instances of an image. They are lightweight and include the application and all its dependencies but share the host’s kernel, which makes them efficient in terms of resources.

   Docker Registry:
   
   The Docker architecture also includes registries, like Docker Hub, where you can store and retrieve images. You can pull images that others have created or push your own images for others to use.
   
   Networking and Storage:
   
   Docker also includes components for networking and storage to enable containers to communicate with each other and the outside world, and to manage data. Networking ensures that the containers can talk to each other and other services, while storage manages the data associated with containers.

4. Creating Docker Images: There are two main methods to create a Docker image:
   • Interactive Method: This method involves running a container from an existing Docker image and then making manual changes to that container’s environment. It’s like setting up a new computer from a base operating system and then installing all the needed software and configurations. Once everything is set up correctly, the resulting state can be saved as a new image.
   • Dockerfile Method: This is a more systematic and reproducible approach where you write a plain-text file known as a Dockerfile. The Dockerfile contains a series of instructions that define how the image should be built. It can include commands to install software, copy files, set environment variables, and other tasks needed to prepare the container. By using a Dockerfile, you can share and rebuild the image easily and consistently.
5. Docker Hub: Docker Hub is a cloud-based service provided by Docker that functions as both a repository and a registry for Docker container images. It plays a central role in the Docker ecosystem by facilitating the storage, sharing, and distribution of container images. Here’s a comprehensive explanation of Docker Hub:
   • Repository Service: Docker Hub serves as a hosted repository where developers and organizations can store their container images. Think of it as a library where you can keep different versions of software packaged in container images.
   • Sharing and Collaboration: One of the key benefits of Docker Hub is the ability to share container images with others. This can include both publicly sharing images with the entire Docker community or privately sharing them within a specific team or organization. It acts as a central location where developers can pull existing images or push their newly created ones, facilitating collaboration and standardization across various stages of development.
   • Docker Registry: Beyond just being a repository, Docker Hub is also a Software as a Service (SaaS) Docker container registry. A registry is like a catalog or index that helps you manage and organize the container images. When you need to find a specific image, the registry helps you locate it within the repository.
   • Public and Private Repositories: Docker Hub offers both public and private repositories. Public repositories are available to anyone, and they often contain commonly used images, such as base operating systems or popular software. Private repositories, on the other hand, are restricted to specific users or teams, providing a secure place to store proprietary or sensitive images.
   • Integration with Docker Tools: Docker Hub is tightly integrated with Docker’s tools, making it easy for users to pull images into their local environment or push their images to the hub. This seamless connection simplifies the workflow, particularly in collaborative or distributed development settings.
6. container OS:
   • Definition and Composition: The container OS is the specific operating system that resides within a container. Unlike traditional virtual machines, where an entire guest OS is installed, the container OS is a lightweight, minimal version of an operating system. It includes only the essential elements needed to run a particular application.
   • Flexibility and Choice: One of the notable features of the container OS is the flexibility it offers. Containers can be packaged with different versions of Linux or Windows OS, depending on the requirements of the application. This allows developers to access specific OS features, utilize particular libraries, or install additional software that the application may need. The choice of the container OS can be tailored to suit the application’s environment and dependencies.
   • Isolation from the Host OS: The container OS operates independently of the host OS. Even though it shares the host’s kernel, it runs in an isolated environment within the container. This isolation ensures that the application’s environment is consistent and controlled, regardless of where the container is deployed. It also adds a layer of security by limiting the container’s access to other parts of the host system.
   • Deployment and Execution: The container OS provides the runtime environment in which the application is deployed and executed. Since it includes all the necessary dependencies and configurations, the application can run consistently across various environments. This makes the container OS an essential part of achieving portability and reproducibility in software deployment.
   • Efficiency and Performance: By focusing only on the necessary components of the operating system, the container OS is generally much smaller and more efficient than a full-fledged OS. This leads to faster startup times and better resource utilization, especially when running multiple containers on a single host.
7. Building Docker Images: You can create a Docker image using a Dockerfile, a plain-text file containing instructions for configuring the container environment. The docker build command builds the image based on the Dockerfile.
   
   Example:
   To build an image from a Dockerfile in the current directory and name it “temp-ubuntu,” you use the command:
   
   

   docker build -t temp-ubuntu .

   • The “.” at the end specifies the context or path where the Dockerfile is located. In this case, the period means the current directory, so Docker will look in the current directory
   • The -t flag is used to tag the image with a name, allowing you to refer to the image by a chosen name and tag instead of its ID. (The ID can be used directly without a special flag.)
8. Viewing Available Containers: This command lists all containers on your system, including those that are running, stopped, paused, etc. Containers are running instances of Docker images. Each container runs in an isolated environment and contains everything needed for an application to run.
   
   

   docker ps -a

   • ps: This is a reference to the Unix command “ps,” which stands for “process status.” It’s used to show information about the running processes on a Unix system. In the context of Docker, docker ps is used to show information about the running containers, which can be thought of as isolated processes managed by Docker.
   • -a: This is an option that stands for “all.” When used with the docker ps command, it means that you want to list all containers, not just the ones that are currently running. Without this option, docker ps will only show containers that are currently in the running state. By adding the -a option, you also get information about containers that are stopped, exited, or in any other non-running state.

   So, docker ps -a is a command that shows the status of all containers on the system, regardless of their current state.

9. Listing Docker Images: You can view the available images on your system using the Docker client. This helps you keep track of the images and their versions.
   
   Example:
   To list all the Docker images present on your system, use the command:
   
   

   docker images

10. Removing Docker Images: If you have unnecessary or obsolete images, you can remove them to free up space on your system.
    
    Example:
    To remove a specific image by its name or ID, use the command:
    
    

    docker rmi image_name_or_id

11. Running Docker Images: To start a container, run the docker run command, specifying the image to run with its name or ID. Once the image is specified to run, Docker finds the image, loads the container from the image, and executes the command specified as the entry point. It’s at this point that the container is available for management and launches from the image.
    
    Additionally, The -d option in the docker run command is used to run the container in detached mode. This means that the container runs in the background of your terminal, freeing up the command line for further interaction.
    
    When you run a container without the -d option, the container’s standard output (and potentially standard error) is attached to your terminal. You’ll see logs or any other output from the container’s main process in your terminal window. If the main process doesn’t run in the background itself, this will effectively lock your terminal until the container is stopped.
    
    By running the command docker run -d image_name, you’re telling Docker to start the container in the background, allowing you to continue using your terminal for other commands. This is particularly useful when running services or other long-running processes that you don’t need to interact with directly.”
    
    Example:
    To run a specific image, use the command:

    docker run -d image_name

12. Pause a container: Pausing a container will suspend all processes. This command enables the container to continue processes at a later stage. The docker unpause command un-suspends all processes in the specified containers.
    
    

    docker pause container_name

13. Restart a container: The container receives a stop command, followed by a start command. If the container doesn’t respond to the stop command, then a kill signal is sent.
    
    

    docker restart container_name

14. Stop a container: To stop a running container, run the docker stop command.
    
    

    docker stop container_name

15. Remove a container: After you remove the container, all data in the container is destroyed.
    
    

    docker rm container_name

16. Communication Between Client and Server: The Docker architecture operates as a client-server implementation. The Docker client sends queries and commands to the Docker server (or daemon), which performs the actual tasks.
    • Client: The command-line interface (CLI) or graphical user interface (GUI) that you use to interact with Docker.
    • Server: The background service that manages all aspects of Docker, including building, running, and orchestrating containers.
17. How to manage Docker containers:
    
    
    • To place a container in the run state, use the run command. You can also restart a container that is already running.
    • A container is considered in a running state until it’s either paused, stopped, or killed. A container can self-exit when the running process completes, or if the process goes into a fault state.
    • To pause a running container, use the pause command. To stop a running container, use the stop command. To send a kill signal if you need to terminate the container, use the kill command.
    • Lastly, to remove containers that are in a stopped state, use the remove command. After removing a container, all data stored in the container gets destroyed.
18. Bridge network:
    • The default Docker network configuration allows for the isolation of containers on the Docker host. This feature enables you to build and configure apps that can communicate securely with each other.
    • The bridge network is the default configuration applied to containers. This network is an internal, private network used by the container, and isolates the container network from the Docker host network.
    • Each container in the bridge network is assigned an IP address and subnet mask with the hostname defaulting to the container name. Containers connected to the default bridge network are allowed to access other bridge connected containers by IP address.
    • Bridge networks apply to containers running on the same Docker server host.
19. Host network: The host network enables you to run the container on the host network directly. This configuration effectively removes the isolation between the host and the container at a network level.
20. None network: To disable networking for containers, use the none network option.
21. Image Registries:
    
    
    
    The concept of image registries in the world of containerization, particularly in relation to Docker, is akin to a library of images. It plays a vital role in the development, deployment, and scaling of applications. Let’s delve into this concept and understand its various facets.
    
    What is a Registry?:
    A registry is a centralized repository where developers can store and manage Docker images. These images are like blueprints that contain the necessary code, libraries, and dependencies needed to run an application inside a container.
    
    Why is a Registry Needed?:
    Registries provide a systematic way to share, distribute, and manage images. Whether deploying a single container or scaling up to use orchestration tools, the registry acts as the go-to place for pulling the required images. This ensures consistency and efficiency, especially in production environments.
    
    Public and Private Registries:
    • Docker Hub: Docker maintains a public registry known as Docker Hub. It’s like a massive public library where developers can share and access a wide variety of images, both official and community-driven.
    • Vendor-Specific Registries: Other vendors, like Azure, may have their own registries tailored to their ecosystems. For example, Azure Container Registry integrates seamlessly with Azure services.
    • Private On-Premises Registries: Enterprises often require more control over their images, especially for security and compliance reasons. A private registry, hosted on the company’s premises, provides this control. It ensures that only authorized users can access and modify the images, protecting sensitive information.

    Deploying to Production Orchestrators:
    When deploying applications in production, especially using orchestrators like Kubernetes, the registry becomes an essential part. The orchestrator pulls the necessary images from the specified registry, ensuring that the correct versions are used and that the deployment is uniform across the entire infrastructure.
    
    Role in the Development Lifecycle:
    Registries facilitate collaboration among development teams. They can store different versions of images, allowing for testing, development, and production uses. A developer can push a newly created or updated image to the registry, and others can pull it for their work, ensuring that everyone is on the same page.
    
    Flexibility and Choices: The availability of public, private, and vendor-specific registries provides flexibility. Developers and organizations can choose the best fit for their needs and even utilize multiple registries simultaneously.

22. Dockerfile:
    
    
    • A Dockerfile is a text file that defines a Docker image. You’ll use a Dockerfile to create your own custom Docker image, in other words to define your custom environment to be used in a Docker container.
    • You’ll want to create your own Dockerfile when existing images don’t satisfy your project needs.
    • A Dockerfile is a step-by-step definition of building up a Docker image. The Dockerfile contains a list of instructions that Docker will execute when you issue the docker build command.
23. Some of the essential instructions in a Dockerfile:
    
    
    • FROM Instruction: This is the foundation of your Docker image. Think of it as choosing the underlying operating system or platform upon which everything else will be built. It might be a particular version of Linux or another lightweight OS, and it creates a layer of dependencies. By choosing this base, you define the environment where your application will live.
    • RUN Instruction: This instruction is about setting up everything your application needs to function. Whether it’s installing software packages, compiling code, or configuring settings, the RUN instruction takes care of preparing the environment. It’s like furnishing an apartment with everything needed to live there, from the furniture to the utensils.
    • COPY Instruction: Sometimes you’ll have files on your local computer that need to be inside the container. The COPY instruction takes care of this, moving files from the Docker client’s current directory into the image. It might be your application’s code, configuration files, or anything else needed at runtime.
    • EXPOSE Instruction: Applications often communicate over networks, whether it’s a web server listening for incoming requests or a database waiting for queries. The EXPOSE instruction tells Docker that the container will be listening on a specified network port when it’s running. It’s like telling your friends your address so they can come to visit; other systems need to know where to find your container.
    • CMD Instruction: Finally, the CMD instruction defines what should happen when the container starts running. It specifies the command to run within the container, essentially giving it its main purpose or function. It might be starting a web server, running a script, or whatever the primary task of the container is. Think of it as the main job or function of the resident living in the apartment.

    In summary, a Dockerfile is a series of steps, like building an apartment from the ground up. You choose the location and type of building (FROM), furnish and equip it (RUN), move in personal belongings (COPY), tell people where it is (EXPOSE), and define the resident’s daily routine (CMD). Each step is crucial in defining what the container is, what it has, and what it does, culminating in a unique and functional environment for running an application.

Commands

1. docker images: Lists all available images on the local system.
2. docker run -d -p <host-port>:<container-port> <image-name-or-id>: Runs a container in detached mode (-d) from the specified image, mapping a port in the container to a port on the host.
3. docker ps: Lists all running containers, including their container IDs, names, and other information.
4. docker rmi <image-name-or-id>:<tag>: Removes the specified image from the local system.
5. docker exec -it <container-id-or-name> /bin/bash: Executes an interactive shell (/bin/bash) inside the specified running container, allowing you to run commands inside the container. (Enter the image)
6. docker ps -l: Lists the last created container, including its container ID, name, and other information.
7. docker commit <container-id-or-name> <new-image-name>:<tag>: Creates a new image from a container’s changes. The new image will have the name and tag specified.
8. docker tag <source-image>:<tag> <target-image>:<new-tag>: Creates a tag <target-image>:<new-tag> that refers to <source-image>:<tag>, allowing you to refer to the image by a new name or version.
9. docker logs <container-id-or-name>: Fetches the logs of a container, allowing you to see the output and any error messages from the main process running inside the container.
10. sudo apt install apt-transport-https ca-certificates curl software-properties-common: This command installs several prerequisite packages needed to enable the use of packages over HTTPS, as well as some common software properties tools. It’s part of the preparation for adding a new repository and ensuring secure communication with it.
11. curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg –dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg: This command downloads the GPG key used to verify the authenticity of packages from Docker’s repository, then adds it to the system’s list of trusted keys. It ensures that the packages you download are the ones published by Docker and haven’t been tampered with.
12. echo “deb [arch=$(dpkg –print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable” | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null: This command adds Docker’s repository to the list of sources that APT will use to find packages. It specifies the architecture, signing key, and the distribution’s codename to tailor the source for your system.
13. sudo apt update: After adding a new repository, this command updates the package database, ensuring that APT knows about the latest versions of packages available from all configured repositories, including Docker’s.
14. apt-cache policy docker-ce: This command displays the installed and candidate versions for the docker-ce package, along with information about the repositories that provide those versions. It’s useful for seeing what versions of Docker are available to install or upgrade to.
15. sudo apt install docker-ce: This command installs the docker-ce package, which provides Docker Community Edition. If there are dependencies that need to be installed along with Docker, this command will handle that as well.
16. sudo systemctl status docker: After installing Docker, this command checks the status of the Docker service. It will tell you whether the service is active, inactive, or has encountered an error. (Check if Docker is running)
17. docker: Typing docker on its own displays help information about the Docker command-line tool, including a list of available subcommands and common options. (Check if docker command line works)
18. whoami: This command is not specific to Docker; it’s a general Unix command that prints the username of the current user. It can be useful to verify that you’re running commands as the expected user, especially when dealing with permissions.
19. sudo groupadd docker: This command creates a new user group named “docker” on the system. By adding users to this group, you grant them permission to run Docker commands without needing root privileges.
20. sudo usermod -aG docker [username]: This command adds the user (in this case, “ubuntu”) to the “docker” group. The -aG option appends the user to the group without removing them from other groups.
21. newgrp docker: This command logs the user into the “docker” group. It can be used after adding a user to the group so that the user can run Docker commands without having to log out and back in.
22. docker run hello-world: This command runs a container using the “hello-world” image, a simple image provided by Docker that prints a welcome message. It’s often used as a test to ensure that Docker is installed and functioning correctly.
23. docker search httpd: You can search for images available on Docker Hub by using the docker command with the search subcommand. For example, this is to search for the Apache httpd image. This would return all of the images that contain the name “httpd”.
24. docker pull httpd: This command downloads the “httpd” image from the Docker Hub repository. This image is used to run Apache HTTP Server in a container.
25. docker images: This command lists all the images available on the local system, including the newly pulled “httpd” image.
26. docker run -d –name docker-apache -p 8080:80 httpd: This command runs the “httpd” image in a container, in detached mode (-d). It names the container “docker-apache” and maps port 80 in the container to port 8080 on the host machine.
27. sudo apt install w3m: This command installs “w3m,” a text-based web browser. It’s a useful tool for accessing web content from the command line.
28. w3m http://localhost:8080: This command opens the “w3m” terminal web browser and navigates to “http://localhost:8080,” where the Apache server running in the Docker container can be accessed.
29. docker stop [container_ID_or_name]: This command stops a running container, where is replaced by the container ID or name. It sends a SIGTERM signal, followed by a SIGKILL if the container doesn’t stop within the timeout.

Infotech

• Tweet
• LINE
• facebook