{"id":1623,"date":"2022-05-31T08:42:18","date_gmt":"2022-05-31T06:42:18","guid":{"rendered":"https:\/\/zen-cori.138-201-132-86.plesk.page\/?p=1623"},"modified":"2022-09-28T16:19:11","modified_gmt":"2022-09-28T07:19:11","slug":"moving-automotive-embedded-software-development-to-the-cloud","status":"publish","type":"post","link":"https:\/\/www.btc-embedded.jp\/ja\/moving-automotive-embedded-software-development-to-the-cloud\/","title":{"rendered":"Moving Automotive Embedded Software Development to the Cloud"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"1623\" class=\"elementor elementor-1623\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3b5f0493 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3b5f0493\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-559a247a\" data-id=\"559a247a\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-d83c847 elementor-widget elementor-widget-text-editor\" data-id=\"d83c847\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.20.0 - 26-03-2024 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p>I will spare you from a lengthy monolog on why a CI Pipeline that requires manually configured environments is fairly limited when it comes to scaling it out to other projects, other teams. I assume that you have felt the pain yourselves.<\/p><p>There are different aspects to this topic including integrating with a cloud provider (AWS, Azure, GCP), using kubernetes or managing the Jenkins controller itself in a new way (JCasC, Jenkins Operator, etc.). However, I want to focus on Jenkins Agents and show you how to move from manually configured VMs to all-purpose agents with Docker. And since a lot of tooling in the automotive embedded world is Windows-based, I will focus on Docker for Windows (for which it\u2019s a lot harder to find decent tutorials!).<\/p><p>In essence, this article aims to help you achieve the ability to design Jenkins Pipelines that can run on any Jenkins Agent. In turn, this enables self-service CI, allowing your team to kick things off with zero delay:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-883721a elementor-widget elementor-widget-image\" data-id=\"883721a\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.20.0 - 26-03-2024 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"768\" height=\"330\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_Agents-768x330.png\" class=\"attachment-medium_large size-medium_large wp-image-6047\" alt=\"\" srcset=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_Agents-768x330.png 768w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_Agents-1536x660.png 1536w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_Agents.png 1880w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">classic approach<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6e6dea4 elementor-widget elementor-widget-image\" data-id=\"6e6dea4\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"768\" height=\"330\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_DockerAgent-768x330.png\" class=\"attachment-medium_large size-medium_large wp-image-6044\" alt=\"\" srcset=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_DockerAgent-768x330.png 768w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_DockerAgent-1536x660.png 1536w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Blog_Jenkins_DockerAgent.png 1880w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">container-based approach<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7c099a8 elementor-widget elementor-widget-heading\" data-id=\"7c099a8\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.20.0 - 26-03-2024 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">What we have<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a1ff07f elementor-widget elementor-widget-text-editor\" data-id=\"a1ff07f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>I will start off with a Jenkins setup with one manually configured agent. This agent has all the software installed to support my workflow. For Model-based Development of embedded software, these are tools like<\/p><ul><li>MATLAB Simulink<\/li><li>An auto-code generator like dSPACE TargetLink<\/li><li>A host compiler<\/li><li>Tools for model-guideline-checking, MISRA-rule-checking and static analysis like Simulink Checks or Polyspace<\/li><li>A test tool like BTC EmbeddedPlatform<\/li><\/ul><p>Our existing Jenkins Pipeline just checks out files from the git repo and runs the tests:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-19ece90 elementor-widget elementor-widget-image\" data-id=\"19ece90\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"413\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-5-1920x1920-f09.png\" class=\"attachment-large size-large wp-image-1631\" alt=\"\" srcset=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-5-1920x1920-f09.png 824w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-5-1920x1920-f09-768x396.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Existing CI Pipeline<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-24084d2 elementor-widget elementor-widget-text-editor\" data-id=\"24084d2\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The node section of the scripted Pipeline specifies a label expression that reflects the different dependencies to applications and OS. This is required by the BTC Test stage that runs the unit tests. Let\u2019s assume for a moment that we already have a docker container image that contains the required applications for each of these stages. We could restructure our Pipeline like this:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c10a0d4 elementor-widget elementor-widget-image\" data-id=\"c10a0d4\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"404\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-6-1920x1920-f09.png\" class=\"attachment-large size-large wp-image-1634\" alt=\"\" srcset=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-6-1920x1920-f09.png 838w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-6-1920x1920-f09-768x388.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Container-based CI Pipeline<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-27e953f elementor-widget elementor-widget-heading\" data-id=\"27e953f\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">What we need<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6cdaf06 elementor-widget elementor-widget-text-editor\" data-id=\"6cdaf06\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The above Pipeline would fail because we have not yet prepared our Jenkins setup to work with Docker and we have not created the mentioned image (ci-unittest). We now need to install the Docker Pipeline plugin on our Jenkins Controller and configure an Agent to host windows-based Docker containers.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8f3060b elementor-widget elementor-widget-heading\" data-id=\"8f3060b\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Preparations<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6ec190f elementor-widget elementor-widget-text-editor\" data-id=\"6ec190f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>In the web interface of the Jenkins Controller: Manage Jenkins &gt; Manage Plugins &gt; Available: we search for docker pipeline and install it. If you already have a Docker registry to host your Docker images you can configure it in the main configuration section (Manage Jenkins &gt; Configure System). For now, it\u2019s not mandatory though because we will use the same machine to build the images and run the containers so we don\u2019t necessarily need a registry.<\/p><p>I\u2019m assuming that you generally know how to connect a machine to serve as a Jenkins agent. If you don\u2019t: checkout my video on the topic:\u00a0<a href=\"https:\/\/youtu.be\/r2u1oYJVkPY\" target=\"_blank\" rel=\"noopener\">https:\/\/youtu.be\/r2u1oYJVkPY<\/a><\/p><p>To keep things simple, we will use a permanent agent to serve as the Docker host. In the Jenkins web interface, go to Manage Jenkins &gt; Manage Nodes and Clouds &gt; New Node. Provide a name and select Permanent Agent. You can adapt this further if you wish but for our purposes, it\u2019s sufficient to specify a label (e.g., docker), a remote root directory (all job workspaces will reside there) and a connection method.<\/p><p>A note on the remote root directory: when Jenkins runs certain parts of a Pipeline in docker, it mirrors the workspace into the container so that the files are available. This is done using the docker run -v flag (&#8211;volumes). In my personal experience this only works robustly if the docker daemon and the mirrored directories are on the same drive (i.e., the C-drive unless you want to reconfigure the docker daemon).<\/p><p>The agent can be a fresh Windows 10 machine. All you need is Docker (<a href=\"https:\/\/docs.docker.com\/docker-for-windows\/install\/\" target=\"_blank\" rel=\"noopener\">https:\/\/docs.docker.com\/docker-for-windows\/install<\/a>) and the windows \u201cContainers\u201d feature. If you don\u2019t have an easy way to get a machine with Windows 10, you can consider using your laptop (for testing purposes, not for production scenarios!).<\/p><p>Once we have connected our new agent to the Jenkins controller, we can run a small pipeline to see if it\u2019s working:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f72eab6 elementor-widget elementor-widget-image\" data-id=\"f72eab6\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"337\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Bildschirmfoto-2022-06-16-um-09.03.23.png\" class=\"attachment-large size-large wp-image-1637\" alt=\"\" srcset=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Bildschirmfoto-2022-06-16-um-09.03.23.png 1098w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Bildschirmfoto-2022-06-16-um-09.03.23-768x323.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e4a4a4c elementor-widget elementor-widget-image\" data-id=\"e4a4a4c\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"730\" height=\"384\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-4-1920x1920-f09-2.png\" class=\"attachment-large size-large wp-image-1643\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-95f5f80 elementor-widget elementor-widget-text-editor\" data-id=\"95f5f80\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>This is looking good, let\u2019s build the images that we need for our existing Jenkins Pipeline.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-da6f93a elementor-widget elementor-widget-heading\" data-id=\"da6f93a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Writing the Dockerfile<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fa217fc elementor-widget elementor-widget-text-editor\" data-id=\"fa217fc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>This part heavily depends on the applications that you want to package into a container image. On an abstract level it always the same though:<\/p><ol><li>Write a Dockerfile, that describes how the container image shall be prepared<\/li><li>Call the docker build command to build the image (and optionally docker push to push it to a registry)<\/li><\/ol><p>This blog article is no substitute for learning how a Dockerfile works but I would like to share some of my insights when it comes to bringing applications to docker that were not designed for it. Mostly, because there\u2019s a couple of things that I wish I\u2019d have known up-front. For more general details, please refer to the official docs (<a href=\"https:\/\/docs.docker.com\/engine\/reference\/builder\" target=\"_blank\" rel=\"noopener\">https:\/\/docs.docker.com\/engine\/reference\/builder<\/a>) and best practices (<a href=\"https:\/\/docs.docker.com\/develop\/develop-images\/dockerfile_best-practices\" target=\"_blank\" rel=\"noopener\">https:\/\/docs.docker.com\/develop\/develop-images\/dockerfile_best-practices<\/a>).<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7303e71 elementor-widget elementor-widget-heading\" data-id=\"7303e71\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Base Image<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-79c29f9 elementor-widget elementor-widget-text-editor\" data-id=\"79c29f9\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>FROM mcr.microsoft.com\/windows\/servercore:20H2<\/p><ul><li>You always start by extending a base image<\/li><li>Microsoft mainly provides 3 base images: windows (Windows 10), servercore (Windows Server) and nanoserver (minimalist edition, very small)<ul><li>o While Nanoserver is very small, it is rarely feasible because you would end up having to add a whole bunch of dependencies to make things work as expected<\/li><li>o The Windows image comes with the highest chances of having everything an application needs but is also fairly big<\/li><li>o I\u2019ve mostly seen Servercore where possible and Windows being used<\/li><\/ul><\/li><li>The part after the : is the version and refers to the major windows update versions such as 1809, 2004 and nowadays 20H2, 21H1, etc.<\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cf6fff1 elementor-widget elementor-widget-heading\" data-id=\"cf6fff1\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">COPY and ADD<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ef2476b elementor-widget elementor-widget-text-editor\" data-id=\"ef2476b\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>You can use COPY to copy files\/folders from your build context (i.e., the directory of your Dockerfile or below) into the container or use add to download something directly and follow it up by unzipping for example:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-46ac4dd elementor-widget elementor-widget-heading\" data-id=\"46ac4dd\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">COPY and ADD<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a4ed432 elementor-widget elementor-widget-image\" data-id=\"a4ed432\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"165\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Bildschirmfoto-2022-06-16-um-09.12.22.png\" class=\"attachment-large size-large wp-image-1646\" alt=\"\" srcset=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Bildschirmfoto-2022-06-16-um-09.12.22.png 1452w, https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/Bildschirmfoto-2022-06-16-um-09.12.22-768x159.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-295275b elementor-widget elementor-widget-heading\" data-id=\"295275b\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">SHELL and RUN<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-190a8e8 elementor-widget elementor-widget-text-editor\" data-id=\"190a8e8\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The RUN command let\u2019s you run any kind of command inside the current container level using the default shell (cmd \/S \/C). You can run powershell commands by simply refering to POWERSHELL as the first argument of your RUN command (as seen above) or by specifying it using the SHELL command:<\/p><p><code>SHELL [\"POWERSHELL\", \"-command\"]<\/code><\/p><p>Installing an application can be as simple as copying the required files to a location (works for the MINGW64 compiler for example). If that\u2019s not possible you can use the applications silent installation mechanism which uses a text file (recorded from a GUI-based installation or created from a template) as input for the installer. If you are having trouble with silent installation processes instantly exiting instead of waiting for completion, I recommend to try something like this (powershell script):<\/p><p><code>$p = Start-Process 'setup.exe' -ArgumentList '\/s' -Wait -PassThru<\/code><\/p><p><code>$p.WaitForExit()<\/code><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9a9f5a0 elementor-widget elementor-widget-heading\" data-id=\"9a9f5a0\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">ENTRYPOINT<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-be1858c elementor-widget elementor-widget-text-editor\" data-id=\"be1858c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The ENTRYPOINT line tells the container runtime what to do when the container starts up. This should start an executable that doesn\u2019t exit immediately, otherwise the container will exit as well.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-46bd8d4 elementor-widget elementor-widget-heading\" data-id=\"46bd8d4\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Remove obsolete installation files<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-debc5f5 elementor-widget elementor-widget-text-editor\" data-id=\"debc5f5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Keeping a container image as small as possible is important, therefore the container image should only contain essential files. Anything that you COPY \/ ADD to the container and only need it temporarily for installation should be removed as part of the Dockerfile. This way, the resulting image will not contain these obsolete files. For example, you can store all of your installation files and scripts in C:\\Temp and remove the complete folder when it\u2019s no longer needed:<\/p><p><code>RUN POWERSHELL Remove-Item -Path C:\/Temp -Force -Recurse<\/code><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3ca3ae4 elementor-widget elementor-widget-heading\" data-id=\"3ca3ae4\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">ENV and Windows Registry<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3fae91d elementor-widget elementor-widget-text-editor\" data-id=\"3fae91d\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Required environment variables can be added with the ENV command, e.g.:<\/p><p><code>ENV MW_MINGW64_LOC C:\/mingw64<\/code><\/p><p>In case you need to add something to the registry, e.g., to configure a license server, you can use the ADD or IMPORT commands of reg.exe. You can then either add the registry key directly or export a *.reg file from somewhere, COPY it to the container and add it:<\/p><p><code>RUN REG ADD \"HKEY_CURRENT_USER\\SOFTWARE\\FLEXlm License Manager\" \/v MY_APP_LICENSESERVER \/t REG_SZ \/d 27000@192.168.0.1<\/code><\/p><p>or<\/p><p><code>RUN REG IMPORT C:\\Temp\\myRegistryEntry.reg<\/code><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-91ba293 elementor-widget elementor-widget-heading\" data-id=\"91ba293\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Debugging \/ Extended analysis<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5aba506 elementor-widget elementor-widget-text-editor\" data-id=\"5aba506\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>In case something doesn\u2019t work as expected or you feel like there is no other way than to manually install something, after building the image you can use<\/p><p><code>docker run -it --entrypoint cmd my-image<\/code><\/p><p>to bring up the container, perform some changes manually and then call exit to stop the container. Afterwards, you can use the container ID (see docker ps -a) to commit the changes that you made into a new image:<\/p><p><code>docker commit deddd08923 my-modified-image<\/code><\/p><p>Please note that I do not recommend to do this outside of very special scenarios. Normally, all that is needed to produce a docker image should be part of the Dockerfile and its context.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bebf125 elementor-widget elementor-widget-heading\" data-id=\"bebf125\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Docker build<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d0d53dd elementor-widget elementor-widget-text-editor\" data-id=\"d0d53dd\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>In essence, your Dockerfile should look somewhat like this with the main content being COPY or ADD to bring files into the container and RUN to perform installations and prepare everything:<\/p><p>\u00a0<\/p><p><code>FROM mcr.microsoft.com\/windows\/servercore:20H2<\/code><\/p><p><code># add required applications &amp; dependencies using ADD, COPY &amp; RUN<\/code><\/p><p><code># in my case Matlab, MINGW, BTC, \u2026<\/code><\/p><p><code>ENTRYPOINT myApp.exe<\/code><\/p><p>\u00a0<\/p><p>Now we can build it using the docker command line. Let\u2019s cmd or powershell on the windows machine that we recently connected as a new Jenkins agent and navigate (cd) into the directory of the Dockerfile. Now we can call:<\/p><p><code>docker build . -t ci-unittest<\/code><\/p><p>No need to push the image into a registry, since they will be hosted on the same machine only (otherwise you would call docker image tag and docker push after the build).<\/p><p>All set, the image is ready to be used by our restructured Pipeline.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3edd05c elementor-widget elementor-widget-heading\" data-id=\"3edd05c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">What we gained<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ece022a elementor-widget elementor-widget-text-editor\" data-id=\"ece022a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>In our previous setup we needed to request an agent that met the needs of our Pipeline. This takes time and resources (as in: people had to spent time to deal with such a request). It also leads to long iterations if the environment is not setup 100% correctly, which can easily happen when we consider the complexity of the tool installations. By using Docker instead, we rely on an immutable image. We only have to get it right once, afterwards it\u2019s guaranteed to work. Updates and changes are automatically documented due to the configuration as code approach (i.e., the Dockerfile).<\/p><p>By removing the dependency to specific agents, we have decoupled the individual Jenkins Pipeline from the environment that is required to run it. Now, all we need to do to get CI support for a project or component is to add a Jenkinsfile to the repository and point Jenkins to the repository URL:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-aadfa40 elementor-widget elementor-widget-image\" data-id=\"aadfa40\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"603\" height=\"831\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-7b-1920x1920-85f.png\" class=\"attachment-large size-large wp-image-1649\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d88eb76 elementor-widget elementor-widget-heading\" data-id=\"d88eb76\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Self-Service CI<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e9bd924 elementor-widget elementor-widget-text-editor\" data-id=\"e9bd924\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>This creates a multi-branch pipeline for the selected git repository based the Jenkinsfile in your repo:<\/p><figure class=\"content-image \"><\/figure>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8d96ccc elementor-widget elementor-widget-image\" data-id=\"8d96ccc\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"508\" height=\"100\" src=\"https:\/\/www.btc-embedded.jp\/wp-content\/uploads\/2022\/05\/cloud-blog-jenkins-agents-8-1920x1920-85f.png\" class=\"attachment-large size-large wp-image-1652\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3810d50 elementor-widget elementor-widget-heading\" data-id=\"3810d50\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Conclusion: All smiles and sunshine?<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6bb1d04 elementor-widget elementor-widget-text-editor\" data-id=\"6bb1d04\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Although we can already benefit from the achievements in cloud technologies, containers and distributed computing, there are still a couple of bumps in the road as of today (May 2021). Many tools are Windows-based and don&#8217;t ship with a system-independent API (such as REST). In our current approach, we throw a bunch of big applications into a single container. This provides an important level of abstraction but does not quite meet the idea of a services-based architecture that is meant to be used in the cloud. For a multi-container setup, communicating with applications across the borders of a container is vital for reliability and performance while keeping the cost low.<\/p><p>Furthermore, many consider windows-based containers more of a bridge-technology and only plan to use it, while waiting for all of their tools to run on a Linux environment. This is reflected in the much lower number of tutorials and docs for windows-based containers. From the technology standpoint, the\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/virtualization\/windowscontainers\/manage-containers\/hyperv-container\" target=\"_blank\" rel=\"noopener\">Process Isolation approach<\/a>\u00a0provides similarly high performance and low resource overhead compared to Linux containers but we still face limitations from time to time that make the life of a DevOps Engineer much harder on Windows.<\/p><p>However, I personally believe that this is a road worth taking despite the bumps we\u2019ll have to face. There is definitely a learning curve to a cloud-based setup (infrastructure management, cost planning and predictions, monitoring, etc.) independent of the details of each container but it\u2019s an inevitable effort and an invest that pays off.<\/p><p>The shift from the current culture of &#8220;request &gt; carefully consider &gt; approve &gt; provision&#8221; to &#8220;self-service + monitored&#8221; is an important part of success and is essential for removing friction and effectively reducing costs. While it is enabled by the technology (cloud, containers) it needs to be supported on the highest management level to be effective. One side effect is, that the daily routine of IT teams will change drastically which will inevitably lead to a certain level of resistance but I sincerely believe that it\u2019s a win, also for IT teams. Honestly \u2013 who wants to spend their time manually configuring the 100<sup>th<\/sup>\u00a0virtual machine for a team, knowing that they surely won\u2019t get notified when it is no longer needed?<\/p><p>All in all, the benefits of a cloud- and container-based infrastructure and a self-service culture are extraordinary and the competitive advantage has already been proven in other industries. When looking at the complex, heterogeneous tool landscapes of automotive embedded software development for example, some doubt that these \u201coverly hyped web development technologies\u201d can apply to their domain. I say: don\u2019t be afraid to dip a foot into the water, it\u2019s already warmer than you\u2019d have thought!<\/p><figure class=\"content-image \"><\/figure>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>I will spare you from a lengthy monolog on why a CI Pipeline that requires manually configured environments is [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":9008,"comment_status":"open","ping_status":"closed","sticky":false,"template":"elementor_theme","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[55,56,54],"product":[],"use_cases":[],"class_list":["post-1623","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-ci-cd","tag-cloud","tag-jenkins"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/posts\/1623","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/comments?post=1623"}],"version-history":[{"count":38,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/posts\/1623\/revisions"}],"predecessor-version":[{"id":9086,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/posts\/1623\/revisions\/9086"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/media\/9008"}],"wp:attachment":[{"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/media?parent=1623"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/categories?post=1623"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/tags?post=1623"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/product?post=1623"},{"taxonomy":"use_cases","embeddable":true,"href":"https:\/\/www.btc-embedded.jp\/ja\/wp-json\/wp\/v2\/use_cases?post=1623"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}