[{"content":"I have been running Omarchy by the famous Ruby on Rails creator dhh as my daily driver for a couple of months now.\nComing over from Bluefin, a Fedora Silverblue based distribution, I was scared of the stability issues that might entail, from running an Arch based distribution. Bluefin\u0026rsquo;s reliability has been rock solid due to it\u0026rsquo;s rpm-ostree(now bootc) based root filesystem management. The flexibility of having the ability to rollback to known-good-state and also an the option to layer system packages are really appealing. That Bluefin was my daily ever since it\u0026rsquo;s relaunch in 2024.\nAs Omarchy was getting all the buzz, I wanted to try it out, and also wanted to start some projects that interface with dbus, varlink, tpm2 chips, and some microcontrollers device connected to my laptop, which has been a bit unweidly to handle with Bluefin\u0026rsquo;s container development paradigm. Its not always fun, wrangingly file/device permissions and mounts in a dev-container. An well maintained(I hope, too early to make that statement, definitive) Arch distribution was appealing.\nI also wanted to check out hyrland. I have been running sway with waybar on my work laptop, but Ubuntu\u0026rsquo;s hyrland support isn\u0026rsquo;t that great yet.\nWanted to share, how I fixed an issue during omarchy boot, that showed up after I booted my laptop after my labor day vaction.\nHopefully some LLM or human finds this article useful to help solve this issue or a similar one.\nWhy is my laptop screen black? I keep my laptop at my desk, connected by usb-c to a dock charging when the monitors on, and not charging when they are not, sometimes I remember to turn my laptop off, I guess this time, I didn\u0026rsquo;t and went for vacation.\nI plugged it in, turned it on, saw the omarchy plymouth boot splash screen for Luks unlock, entered my password. Nothing. Blank screen, with a cursor.\nKnowing the laptop was still powered on, my experience stemmed the thought that, initrd transition to rootfs hung up, maybe?\nPressed ctrl-alt-f2 to get a tty, gladly I could see a login prompt, with my username, not an emergency shell drop to the root. This gave me confidence that it\u0026rsquo;s not an issue in the initrd, and probably I am in the rootfs as I can seem my local user.\nDigging for clues Logged in, first thing I ran\nsudo systemctl --failed In hopes to find, What might have failed that caused graphical.target to fail? Found that, the a service called omarchy-seamless-login.service failed.\nSep 03 18:27:17 devone systemd[1]: omarchy-seamless-login.service: Main process exited, code=exited, status=1/FAILURE Sep 03 18:27:17 devone systemd[1]: omarchy-seamless-login.service: Failed with result \u0026#39;exit-code\u0026#39;. Ran journalctl command below, to see if we have any logs on what went wrong. Cound\u0026rsquo;t find anything usefull only that the process exited, and the auto restart was tried only for it fail again.\nsudo journalctl -xeu omarchy-seamless-login.service Now I wanted to sheck what this systemd service was doing under the hood. Found the service under /etc/systemd/system/(this probably should be under /usr/lib/systemd/ or /usr/share/factory/etc seeing how curcial this for the distribution login, not that it would fix this issue, it would just fit under the Linux FSH, especially if Omarchy could pivot to a Steam OS base with A/B updates, ooh an arch distribution, with the reliability of Steam OS with the user experience of Omarchy would be nice)\nThe service runs this\nExecStart=/usr/local/bin/seamless-login uwsm start -- hyprland.desktop uwsm is a Universal Wayland Session Manager, which seems to be systemd unit generator for wayland sessions.\nAs a trial and hope that the manual run might generate logs, ran it manually. Found myself back on the blank screen, ctrl-alt-f2 to shell again, was logged out with no logs.\nHopping the output was logged to the journal, ran\nsudo journalctl -xb Pressed shift+g, to scroll to the end. Found nothing useful, or relevant.\nIn hopes to find some clues started grep for the keyword error and failed in the journal logs.\nFound these lines.\nSep 03 18:27:17 devone uwsm_hyprland.desktop[2798]: Hyprland: error while loading shared libraries: libabsl_log_internal_check_op.so.2508.0.0: cannot open shared object file: No such file or dire\u0026gt; Sep 03 18:27:17 devone systemd[2341]: wayland-wm@hyprland.desktop.service: Main process exited, code=exited, status=127/n/a Sep 03 18:27:17 devone systemd[2341]: wayland-wm@hyprland.desktop.service: Failed with result \u0026#39;exit-code\u0026#39;. Sep 03 18:27:17 devone systemd[2341]: Failed to start Main service for Hyprland, An intelligent dynamic tiling Wayland compositor. Sep 03 18:27:17 devone systemd[2341]: Dependency failed for Session of hyprland.desktop Wayland compositor. I think we found the problem? Hmm, that seems like an error message could explain a blank screen. Quick google search the libabsl comes from the abseil-cpp.\nInstalled the dependency.\nyay -S abseil-cpp Rebooted\nsudo reboot Voila, we are back in business!\nI can see the beautiful Omarchy hyprland session back again!\n","permalink":"https://vasuper.vasu3797.workers.dev/posts/omarchy-fix/","summary":"\u003cp\u003eI have been running \u003ca href=\"https://omarchy.org/\"\u003eOmarchy\u003c/a\u003e by the famous Ruby on Rails creator \u003ca href=\"https://x.com/dhh/\"\u003edhh\u003c/a\u003e as my daily driver for a couple of months now.\u003c/p\u003e\n\u003cp\u003eComing over from \u003ca href=\"https://projectbluefin.io/\"\u003eBluefin\u003c/a\u003e, a \u003ca href=\"https://silverblue.fedoraproject.org/\"\u003eFedora Silverblue\u003c/a\u003e based distribution, I was scared of the stability issues that might entail, from running an Arch based distribution. Bluefin\u0026rsquo;s reliability has been rock solid due to it\u0026rsquo;s \u003ca href=\"https://docs.fedoraproject.org/en-US/bootc/rpm-ostree/\"\u003erpm-ostree(now bootc)\u003c/a\u003e based root filesystem management. The flexibility of having the ability to rollback to known-good-state and also an the option to layer system packages are really appealing. That Bluefin was my daily ever since it\u0026rsquo;s relaunch in \u003ca href=\"https://www.ypsidanger.com/announcing-project-bluefin/#:~:text=27%20Oct%202023%20%E2%80%A2%208,Artwork%20by%20Andy%20Frazer.\"\u003e2024\u003c/a\u003e.\u003c/p\u003e","title":"Fixing a hyprland shared library issue in omarchy"},{"content":"A tutorial I authored is finally available in the official AWS documentation!!\nhttps://docs.aws.amazon.com/greengrass/v2/developerguide/gg-with-tpm-tutorial.html What is it about? The tutorial shows process of stemming your AWS IoT Thing/Device\u0026rsquo;s Certificates for your AWS Greengrass Nucleus against the Trusted Platform module chip available on a device that supports one.\nWhy would one do this? Let\u0026rsquo;s take this scenario, your business operates a fleet of Cloud native Endpoints (basically computer/devices that talk to the cloud). This could be devices like point of sale systems, embedded devices, industrial PCs, any computer you probably see out in the wild that isn\u0026rsquo;t a personal computer.\nA common mechanism such devices use to authenticate to a cloud control plane is a public key infrastructure standard called x509.\nThis entails the process of creating a public key, a private key, a certificate signing request, and finally a x509 certificate signed by the cloud control plane that is authorized to be used.\nThis certificate is now credentials for the device to talk to the cloud control plane.\nIf a malicious actor(hacker!!!) get\u0026rsquo;s access to our device, remotely, locally, any which way. They can now steal the certificate off the device. With this our hacker now has access to our cloud control plane. While we might have role based access control for our cloud services, that might not be enough to prevent attacks or start a DDOS, and still a person we really don\u0026rsquo;t want has access to something they shouldn\u0026rsquo;t have.\nNow a days you can get devices with a module/chip called the Trusted Platform Module(TPM)(yes, this is the chip that was made mandatory to run Windows 11). This chip is dedicated to run cryptographic operations and can create keys, which is embedded to the motherboard, and acts as a root of trust.\nIn the tutorial above, I show how we can root a private and public key from the TPM chip, and then store a certificate in the TPM for safekeeping. Now our hacker even if he has access to our device can\u0026rsquo;t steal credentials off the device, and even if he can, he can\u0026rsquo;t use it unless he has that exact TPM chip. TPM does not allow keys to be extracted off the chip.\nIf he steals the computer and has time to de-solder the chip and put it on a new computer, then you have physical security problem, I can\u0026rsquo;t help you with that.\nWhat\u0026rsquo;s the tech behind this?\nWhile this tutorial is geared towards AWS IoT Greengrass (as I work there), steps 1-7 can be use for any public private key auth infrastructure for example WiFi that is authenticated with EAP-TLS. The same instructions can be found in this link as well\nhttps://github.com/tpm2-software/tpm2-pkcs11/blob/master/docs/EAP-TLS.md The tutorial uses the tools provided by the tpm2-software project. We use the the PKCS11 interface, and the tpm2_ptool from the tpm2-pkcs11. You can compile these from source, or it\u0026rsquo;s available in most mainstream Linux distributions. In the tutorial I use Ubuntu 24.04.\nShould I know something before implementation this for my edge compute fleet?\nThe TPM chip is not a storage device, it cannot store infinite keys. I discovered, that the TPM can at-most contain 7 key material.\nThis means if you re-provision your device and re-bootstrap your credentials you can\u0026rsquo;t do this more than 7 times.\nFear not, I submitted a fix to the tpm2_ptool, that fixes the destroy command that will clear any keys on the TPM2 chip.\nhttps://github.com/tpm2-software/tpm2-pkcs11/pull/883 Feed the tutorial to the nearest LLM, with these fixes, to create scripts that can automate the bootstrap and teardown process.\nNow you should have a secure endpoint that can also be re-provisioned with no limitations.\nHappy hacking!!\n","permalink":"https://vasuper.vasu3797.workers.dev/posts/secure-gg-with-tpm/","summary":"\u003cp\u003eA tutorial I authored is finally available in the official AWS documentation!!\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://docs.aws.amazon.com/greengrass/v2/developerguide/gg-with-tpm-tutorial.html\"\u003ehttps://docs.aws.amazon.com/greengrass/v2/developerguide/gg-with-tpm-tutorial.html\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eWhat is it about?\u003c/strong\u003e\nThe tutorial shows process of stemming your AWS IoT Thing/Device\u0026rsquo;s Certificates for your AWS Greengrass Nucleus against the Trusted Platform module chip available on a device that supports one.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWhy would one do this?\u003c/strong\u003e\nLet\u0026rsquo;s take this scenario, your business operates a fleet of \u003ca href=\"https://www.truesec.com/service/cloud-native-endpoints\"\u003eCloud native Endpoints\u003c/a\u003e (basically computer/devices that talk to the cloud).  This could be devices like point of sale systems, embedded devices, industrial PCs, any computer you probably see out in the wild that isn\u0026rsquo;t a personal computer.\u003c/p\u003e","title":"My Tutorial for Securing AWS Greengrass with TPM2 is live!"},{"content":"Vasudevan Perumal vasu3797@gmail.com | www.linkedin.com/in/vasuper | 773-707-1927 | vasuper.vasu3797.workers.dev\nExperience System Development Engineer II | Amazon OpsTech \u0026amp; Robotics | Austin, TX | Dec 2023 - Present (1 yr 5 mos)\nLed the development of the OS Bakery, a CI mechanism for OS builds, and a toolchain of dev-tools for local builds. Fully automating git merge to build, and publish of OS artifacts to device deployment. Designed a framework of abstractions for OS customization in the Amazon context, making OS customization accessible to cloud-native engineers, reducing time to prototype for teams, from days to a couple of hours. Spearheaded development of AmazonEdgeOS, first ever custom Linux distribution at Amazon Robotics. Security and reliability as core tenets, we implemented A/B Updates, TPM2 full disk encryption, etc. Designed a virtual OS testing service enabling developers and integration testing frameworks, to spin up test instances, run tests, and teardown instances on-demand, built with libvirt, vagrant, and pytest. Developed abstractions, for teams to leverage TPM2 root of trust for Cloud service and Wifi EAP registration. System Development Engineer I | Amazon OpsTech \u0026amp; Robotics | Austin, TX | Jul 2021 - Dec 2023 (2 yrs 6 mos)\nIntroduced a mechanism to build custom Ubuntu images at Amazon Robotics, enabling teams to “shift-left” their dev and testing process. Leading to a 200% increase in iteration velocity in developing solutions. Introduced Image-based Provisioning at Amazon FCs, reducing device provisioning time from 45 mins to 6 mins. Done through a custom boot initrd, pre-built OS images, with a in-house developed installer. Developed dev-tools, and CI/CD mechanism for internal developer teams to package their code into debian packages, and automating the process from git merge to debian package build and publishing. Helped build a mechanism to vend internal Debian Repos to 200K edge devices. Using AWS Cloudfront CDN. Web Systems Operations Intern | Americaneagle.com | Des Plaines, IL | June 2020 - Dec 2020 (7 mos)\nBuilt a python app translating Cisco LoadBalancer, to A10 LoadBalancer configurations, for a migration initiative. The app automated a 2 month initiative to 2 weeks across our Colo DataCenters. Developed a bash/powershell script to automate clearing RAID config of disks on CentOS 6 servers, reducing SysAdmin’s time to recycle drives from 20 mins to 1 min. Skills Programming Languages: Python, Bash, Rust, Typescript Cloud Tech: AWS Lambda, DynamoDB, API Gateway, Codebuild, Cloudformation, IoT, SSM, Greengrass, SNS, Kinesis, Cloudwatch, CodePipelines, ECS, CloudFront, KMS. Tools, and Platforms: systemd, mkosi, yocto, Containerization(Docker,Podman,systemd-nspawn), Flatpak, Debian/APT package management, btrfs, micro-repo management, Smithy API Framework, HashiCorp Packer and Vagrant, libvirt, qemu, MQTT SDKs for AWS IoT, and Greengrass. Technologies and Concepts: Trusted Platform Module(TPM2), Remote Attestation, Immutable Linux(A/B updates, OsTree), Linux Distribution Management(Ubuntu, Fedora, NixOS), API Design and Data Modelling. Education Masters in Electrical and Computer Engineering | Illinois Tech | Chicago, IL | Aug 2019 - May 2021 Coursework: Advanced Computer Networking, Cybersecurity, Network Security, IoT, Wireless and 5G Networks Bachelor in Electronics and Communication Engineering | SRM IST | Chennai, India | Aug 2015 - May 2019 Coursework: Computer Networking, Communication Design, Mobile Networks, MicroProcessors, and MicroControllers Activities Made upstream contributions to TPM2 TCG Linux project, fixing their PKCS11 teardown process, available now in all major linux distros. Open sourced internal projects, like aws-ready-edge-os, a public fork of AmazonEdgeOS, aws-mkosi-builder, an AWS native OS build Infra library. Ecrmedataextractor, an OCI compliant tool to gather SBOM data. Made minor contributions to monitord, a Dbus based systemd monitoring tool, and mkosi(OS build tool). Attended All Systems Go 2024, foundational linux userspace conference. Member of the pyTexas community. Publications Smartphone APP for Continuous Observation of Pollution Levels Due to Particulate Matter Measured by Laser Mie Scattering Collaborated with 4 researchers in developing a mobile IoT pollution monitoring system that uses Mie Scattering Technique to measure particulate matter-based pollution to gauge living conditions of an area. Publication link here Effects of ambient air pollution on respiratory and eye illness in population living in Kodungaiyur, Chennai Publication link here ","permalink":"https://vasuper.vasu3797.workers.dev/resume/","summary":"\u003ch1 id=\"vasudevan-perumal\"\u003e\u003cstrong\u003eVasudevan Perumal\u003c/strong\u003e\u003c/h1\u003e\n\u003cp\u003e\u003ca href=\"mailto:vasu3797@gmail.com\"\u003evasu3797@gmail.com\u003c/a\u003e | \u003ca href=\"http://www.linkedin.com/in/vasuper\"\u003ewww.linkedin.com/in/vasuper\u003c/a\u003e | 773-707-1927 | \u003ca href=\"https://vasuper.vasu3797.workers.dev/\"\u003evasuper.vasu3797.workers.dev\u003c/a\u003e\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"experience\"\u003e\u003cstrong\u003eExperience\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eSystem Development Engineer II | Amazon OpsTech \u0026amp; Robotics | Austin, TX | \u003cem\u003eDec 2023 - Present (1 yr 5 mos)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eLed the development of the OS Bakery, a CI mechanism for OS builds, and a toolchain of dev-tools for local builds. Fully automating git merge to build, and publish of OS artifacts to device deployment.\u003c/li\u003e\n\u003cli\u003eDesigned a framework of abstractions for OS customization in the Amazon context, making OS customization accessible to cloud-native engineers, reducing time to prototype for teams, from days to a couple of hours.\u003c/li\u003e\n\u003cli\u003eSpearheaded development of AmazonEdgeOS, first ever custom Linux distribution at Amazon Robotics. Security and reliability as core tenets, we implemented A/B Updates, TPM2 full disk encryption, etc.\u003c/li\u003e\n\u003cli\u003eDesigned a virtual OS testing service enabling developers and integration testing frameworks, to spin up test instances, run tests, and teardown instances on-demand, built with libvirt, vagrant, and pytest.\u003c/li\u003e\n\u003cli\u003eDeveloped abstractions, for teams to leverage TPM2 root of trust for Cloud service and  Wifi EAP registration.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eSystem Development Engineer I | Amazon OpsTech \u0026amp; Robotics | Austin, TX |  \u003cem\u003eJul 2021 - Dec 2023 (2 yrs 6 mos)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e","title":"Resume"},{"content":"Cross Account Cross Region(XAXR) Cloudwatch Observability has been a feature that has been around for a while, it was introduced by AWS in 2022.\nLet\u0026rsquo;s consider this scenario, you work in a company, that uses AWS. You are in a team that vends constructs/templates to other internal teams in your company. These internal teams are your customers. You vend your solution as an abstraction, that solves an undifferentiated problem for your customer. This let\u0026rsquo;s your customer focus on what makes their beer taste better.\nLet\u0026rsquo;s also consider a scenario where your team manages a service, and interacts with an other service, and you you want to monitor the health of your dependency.\nXAXR Cloudwatch is a nice way to keep tabs on your solution, for example to gather key business metrics, or help troubleshoot an issue, with your customer. With the second secanrio, XAXR also lets you create event traces between services across accounts. This is how it works.\nSource accounts share their observability data with the monitoring account. The shared observability data can include the following types of telemetry:\nMetrics in Amazon CloudWatch. You can choose to share the metrics from all namespaces with the monitoring account, or filter to a subset of namespaces. Log groups in Amazon CloudWatch Logs. You can choose to share all log groups with the monitoring account, or filter to a subset of log groups. Traces in AWS X-Ray Applications in Amazon CloudWatch Application Insights Monitors in CloudWatch Internet Monitor Here are some more details on this mechanism:\nSource Account sets up a Link to a single or upto 5 Monitoring Accounts. Monitoring Account will setup as a Sink with two types of permissions options. Account IDs AWS Org IDs The monitoring Account can link upto 100,000 source accounts. There are no additional cost implication to either the source account or monitoring account Now let\u0026rsquo;s see some CDK on how to set this up.\n1. Setting up the OAM Link and the XAXR role This would be used to setup permisions in your Source Account:\nimport * as cdk from \u0026#39;aws-cdk-lib\u0026#39;; import * as iam from \u0026#39;aws-cdk-lib/aws-iam\u0026#39;; import * as oam from \u0026#39;aws-cdk-lib/aws-oam\u0026#39;; import { Construct } from \u0026#39;constructs\u0026#39;; export type CloudWatchPolicy = | \u0026#39;CloudWatch-and-AutomaticDashboards\u0026#39; | \u0026#39;CloudWatch-and-ServiceLens\u0026#39; | \u0026#39;CloudWatch-AutomaticDashboards-and-ServiceLens\u0026#39; | \u0026#39;CloudWatch-core-permissions\u0026#39; | \u0026#39;View-Access-for-all-services\u0026#39;; export interface CloudWatchCrossAccountProps { monitoringAccountIds: string[]; policy: CloudWatchPolicy; } export interface CreateOamLinkProps { monitoringAccountId?: string; // Optional - defaults to 637423601711 sinkIdentifier?: string; // Optional - defaults to the one in template labelTemplate?: string; // Optional - defaults to $AccountName } const POLICY_MAP: Record\u0026lt;CloudWatchPolicy, string[]\u0026gt; = { \u0026#39;View-Access-for-all-services\u0026#39;: [ \u0026#39;arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess\u0026#39;, \u0026#39;arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess\u0026#39;, \u0026#39;arn:aws:iam::aws:policy/job-function/ViewOnlyAccess\u0026#39;, \u0026#39;arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess\u0026#39;, ], \u0026#39;CloudWatch-and-AutomaticDashboards\u0026#39;: [ \u0026#39;arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess\u0026#39;, \u0026#39;arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess\u0026#39;, ], \u0026#39;CloudWatch-and-ServiceLens\u0026#39;: [ \u0026#39;arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess\u0026#39;, \u0026#39;arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess\u0026#39;, ], \u0026#39;CloudWatch-AutomaticDashboards-and-ServiceLens\u0026#39;: [ \u0026#39;arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess\u0026#39;, \u0026#39;arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess\u0026#39;, \u0026#39;arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess\u0026#39;, ], \u0026#39;CloudWatch-core-permissions\u0026#39;: [\u0026#39;arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess\u0026#39;], }; export function createCloudWatchCrossAccountRole( scope: Construct, id: string, props: CloudWatchCrossAccountProps, ): iam.Role { const principals = props.monitoringAccountIds.map((accountId) =\u0026gt; new iam.AccountPrincipal(accountId)); const managedPolicies = POLICY_MAP[props.policy].map((policyArn) =\u0026gt; iam.ManagedPolicy.fromAwsManagedPolicyName(policyArn.replace(\u0026#39;arn:aws:iam::aws:policy/\u0026#39;, \u0026#39;\u0026#39;)), ); return new iam.Role(scope, id, { roleName: \u0026#39;CloudWatch-CrossAccountSharingRole\u0026#39;, assumedBy: new iam.CompositePrincipal(...principals), managedPolicies, path: \u0026#39;/\u0026#39;, }); } export function createOamLink(scope: Construct, id: string, props: CreateOamLinkProps = {}): oam.CfnLink { const { monitoringAccountId = \u0026#39;63742\u0026#39;, sinkIdentifier = \u0026#39;arn:aws:oam:us-east-1:6371711:sink/e2aa2952-d9c6-4e40-94e0-31\u0026#39;, labelTemplate = \u0026#39;$AccountEmail\u0026#39;, } = props; // Create condition to skip in monitoring account const skipMonitoringAccount = new cdk.CfnCondition(scope, `${id}SkipMonitoringAccount`, { expression: cdk.Fn.conditionNot(cdk.Fn.conditionEquals(cdk.Stack.of(scope).account, monitoringAccountId)), }); // Create the OAM Link const link = new oam.CfnLink(scope, id, { labelTemplate, resourceTypes: [ \u0026#39;AWS::CloudWatch::Metric\u0026#39;, \u0026#39;AWS::Logs::LogGroup\u0026#39;, \u0026#39;AWS::XRay::Trace\u0026#39;, \u0026#39;AWS::ApplicationInsights::Application\u0026#39;, \u0026#39;AWS::InternetMonitor::Monitor\u0026#39;, ], sinkIdentifier, }); // Apply the condition link.cfnOptions.condition = skipMonitoringAccount; return link; } Link to gist: https://gist.github.com/Vasu77df/22e69620cbcefe824f6d8fdb8809e8f8\nYou can use this in your CDK Construct or Stack\nconst xaxrUwcCentralObeservability = createCloudWatchCrossAccountRole( scope, \u0026#39;BakeryUwcGodViewRole\u0026#39;, crossAccountProps, ); const oamLink = createOamLink(this, \u0026#39;OamLink\u0026#39;, { monitoringAccountId: \u0026#39;6374231\u0026#39;, sinkIdentifier: \u0026#39;arn:aws:oam:us-east-1:63742:sink/e2aa2952-d9c6-4e40-94e0-31c\u0026#39;, labelTemplate: \u0026#39;$AccountEmail\u0026#39;, }); 2. Setting up the Sink Setting up the monitoring account permissions through CDK is a bit tedious and unecessarily complex, as we don\u0026rsquo;t have the Level 2 CDK Constructs yet.\nIf you company or Org has a good standard on AWS Organizational Ids, the most simple process is to just enable it through the AWS Console, and enter you Org ID.\nHere are the instruction from the AWS Documentation:\nhttps://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Unified-Cross-Account-Setup.html#Unified-Cross-Account-Setup-ConfigureMonitoringAccount 3. Conclusion Now with this setup you can search for metrics across accounts, setup dashboards, with metrics from mulitple accounts, and even see log groups across accounts. A very nice feature is seeing X-Ray traces across resources hosted in multiple accounts.\n4. References https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Unified-Cross-Account-Setup.html https://aws.amazon.com/blogs/aws/new-amazon-cloudwatch-cross-account-observability/ ","permalink":"https://vasuper.vasu3797.workers.dev/posts/xaxr-cloudwatch/","summary":"\u003cp\u003eCross Account Cross Region(XAXR) Cloudwatch Observability has been a feature that has been around for a while, it was introduced by AWS in \u003ca href=\"https://aws.amazon.com/blogs/aws/new-amazon-cloudwatch-cross-account-observability/\"\u003e2022\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eLet\u0026rsquo;s consider this scenario, you work in a company, that uses AWS. You are in a team that vends constructs/templates to other internal teams in your company. These internal teams are your customers. You vend your solution as an abstraction, that solves an undifferentiated problem for your customer. This let\u0026rsquo;s your customer focus on what makes their \u003ca href=\"https://podup.substack.com/p/jeff-bezoss-beer-tasting-analogy\"\u003ebeer taste better\u003c/a\u003e.\u003c/p\u003e","title":"Setting up Cloudwatch Cross Account Observability with CDK"},{"content":"FOSDEM is a free and non-commercial event organised by the community for the community. The full form is Free and Open Source Developers European Meeting, you can learn more about it\u0026rsquo;s history here.\nMy day to day involves working with Linux, and maintaining a Linux Distribution for my company and building new features for the distribution. FOSDEM is great conference for Linux builders and even users to learn about the latest greatest happening in the community.\nI curated a list of talks that I think are relevant to any builder or system administrator, working with Linux, especially for folks working in a similar problem space, as I am.\nTalks systemd keynote 14 Years of systemd Linux distribution engineering Boot from network attached devices using mkosi-initrd (or why systemd distributions should really start considering mkosi-initrd) Fedora Silverblue With Disk Encryption: How I Almost Lost Everything But Gained Much Wisdom (Side Story: Bmaptool And Ddrescue: Why One Should Never Ever Use Dd) Rhino Linux and Pacstall: Towards a Rolling Ubuntu Building the Future: Understanding and Contributing to Immutable Linux Distributions Exploring Open Source Dual A/B Update Solutions for Embedded Linux Image-based Linux and Boot security systemd \u0026amp; TPM in 2025 Lennart Poettering 09:00 09:30 ParticleOS: Can we make Lennart Poettering run an image based distribution?! Daan De Meyer 09:30 10:00 FDE is almost there, how do we tackle the last hurdles? Richard Brown 10:00 10:30 \u0026ldquo;Signed, Sealed, and Delivered\u0026rdquo;, with UKIs and composefs Timothée Ravier, Allison Karlitskaya 10:30 11:00 Lessons learned from deploying boot security features on embedded systems Valentin Geffroy, Johann Gautier 11:00 11:30 Generating immutable, A/B updatable, securely booting Debian images Jan Kiszka 11:30 12:00 Bootable Containers and Image Mode: Transforming Linux OS Management with Bootc Eric Curtin, Pierre-Yves Chibon 12:00 12:30 Case Study: Measured Boot and Remote Attestation in Confidential Containers OS proper Rust for Linux Rewriting the future of the Linux essential packages in Rust ? How to push your testing upstream Enabling Architectural Features in Debian: PAC and BTI on arm64 Redox OS \u0026ndash; a Microkernel-based Unix-like OS Rust for Linux: an overview Multi-Profile UKIs and other ways to supercharge your Unified Kernel Images Endpoint Cloud, and Linux security Remote Attestation in the cloud Remote Attestation on Arm TrustZone OP-TEE with VERAISON Verifier \u0026mdash; current status and future plan \u0026mdash; Measurement and Attestation Schemes for Container Sandboxes Recent TPM Security Enhancements to the Linux Kernel Hardware backed SSH keys: ssh-tpm-agent A retrospective on Google’s SBOM implementation Container technology Running Containers Under Systemd: Exploring Podman Quadlet Immutable All the Way Down - using System Extensions to ship Kubernetes Low level systems AI/ML The Local AI Rebellion Hugging Face ecosystem for Local AI/ ML History and advances of quantization in llama.cpp Building Your (Local) LLM Second Brain Building AI Applications from your desktop with Podman AI Lab Self-hosted LLMs at a scale with Paddler Testing and Infrastructure Streamlining package testing with Molecule and Jenkins KernelCI - upgrading Linux development and integration workflows Refining the Release Strategy of a Custom Linux Distro Running QEMU Inside Browser The IaC Tooling Multiverse and the Future of IaC Python Create A Custom Linux Init in Python Anatomy of a Python OpenTelemetry instrumentation Rust Rust-ifying the Linux kernel scheduler (in user space) Lessons from rewriting systems software in Rust Adventures in oxidizing Arch Linux Package Management ","permalink":"https://vasuper.vasu3797.workers.dev/posts/fosdem-2025/","summary":"\u003cp\u003eFOSDEM is a free and non-commercial event organised by the community for the community. The full form is Free and Open Source Developers European Meeting, you can learn more about it\u0026rsquo;s history \u003ca href=\"https://fosdem.org/2025/about/\"\u003ehere\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eMy day to day involves working with Linux, and maintaining a Linux \u003ca href=\"https://en.wikipedia.org/wiki/Linux_distribution\"\u003eDistribution\u003c/a\u003e for my company and building new features for the distribution. FOSDEM is great conference for Linux builders and even users to learn about the latest greatest happening in the community.\u003c/p\u003e","title":"Curated List of FOSDEM 2025 Talks for Linux Distribution Builders"},{"content":"If you have ever run docker pull or pulled a container using the favorite container runtime tool of your choice, you\u0026rsquo;ve probably interacted with a container registry.\nA container registry does not only provide mechanisms to store and distribute container images, it also exposes APIs that clients can use to perform other actions like fetching the container\u0026rsquo;s metadata without pulling the entire container.\nAt work, I had a problem to solve, which is when I thought why not leverage these APIs to solve my problem. In this post we\u0026rsquo;ll see how I leveraged the Container Registry APIs for my benefit and possibly how you the reader can use it as well.\n1. The problem. We run build jobs of OS images meant for bare-metal edge devices, at work. We package the build job and the OS definitions as a container image and execute them in AWS CodeBuild.\nOur eventual aim is ensure our builds are reproducible. We have made some progress through a versioned build environment i.e the container image, and through making snapshot-ted copies of the upstream package mirrors and locking each OS version to use only the same snapshot.\nThis alone does not guarantee reproducible builds. There are some deeper implications and problems to solve to achieve deterministic reproducible builds. I recommend checking out the Reproducible Builds Orgs website to learn more about the problems and their commandments that every software package has to follow to truly achieve reproducibility.\nhttps://reproducible-builds.org/ https://reproducible-builds.org/docs/commandments/ In the meantime, we want to bridge this gap as best as we can, by collecting the SBOM(Software Bill of Materials) in how many vectors as possible for each OS build. If we can ensure reproducibility, we want to provide traceability for all the components involved in the build. For the OS in itself, we capture the apt package manifest(we build Ubuntu based images) and publish it along with the OS image to our internal artifact registry.\nA data we want to collect is the metadata of the container image that executed the build job, and publish those details as well.\nI set some requirements for myself for mechanism that collects this data.\nI want to capture the details without affecting the OS build times. I want to capture the details without requiring to setup an out of band mechanism that peers into the Build job. Something that is simple and maintainable that can run just before executing the actual build. Avoid having to install the docker client or any other client to gather metadata if possible. With these in-place, my first thought was, why not checkout the registry APIs and see what methods that exposes?\n2. OCI Specifications. Contrary to some folk\u0026rsquo;s assumption Docker is not the only container platform out there. But Docker was the founder of the Specifications body that defines the open source specifications for Containers. Most other container platforms implement this specification, like podman and, kata containers. The Open Container Initiative (OCI) founder by Docker is a Linux Foundation project who\u0026rsquo;s purpose is to define open industry standards around container formats and run-times.\nOCI contains three specifications:\nRuntime Specification Defines the configuration, execution environment, and life-cycle of a container. Image Specification Specifications for tools that build, transport and prepare container images to run. Distribution Specification Defines an API protocol to facilitate and standardize the distribution of content that follow the image specification. For the problem I want to solve, I care about the Image and Distribution Specification.\nIn this post, we will not delve into the details of each field or method the specifications provide, you can dive deep following the links above. I will provide a high level overview on certain items that were pertinent to the problem I was solving.\n2.1 Image specification. Image specification defines 5 components that I care about, and are also fundamental to an OCI defined container image. They are the\nImage Manifest It contains metadata about the contents and dependencies of the image. Image index A higher-level manifest which points to a list of manifests. Image Configuration This includes information like the application arguments, environment config, and defines the layer ordering for the final runtime bundle. File-System Layers: This is the container filesystems defined as a changeset for the image. Descriptor: a reference that describes the type, metadata and the content address of the referenced content. Below are diagrams with a high level grouping of how they come together: Source: https://specs.opencontainers.org/image-spec/?v=v1.0.1#overview\nFor my purposes, I would like to capture the Image manifest and the Image configuration for the Build Container running my OS build.\n2.2 Distribution specification. The Distribution specification provides API specs for the API methods, request, response, and error objects, for defining an HTTP based registry used to distribute container content defined from the Image specification we saw above.\nThe distribution specification defines specification various use cases that you might have used while interacting with a container registry, like\nContent verification through hash digests Resumable Push Resumable pull Layer upload de-duplication. The OCI distribution specification is based on the already existing Docker HTTP API V2 Protocol.\nTo solve my problem, of pulling the container metadata, all I require is methods from the V2 protocol that let me pull the image manifest and configuration.\nThe GET /v2/\u0026lt;name\u0026gt;/manifests/\u0026lt;references\u0026gt; path provides the call to get the image manifest.\nhttps://distribution.github.io/distribution/spec/api/#pulling-an-image-manifest This method will return the Image manifest that contains the Image configuration and file system layers information.\nThe image configurations will contain a digest, this is a unique identifier of Descriptor that let\u0026rsquo;s us identify the location of the Image configuration that we can pull, to capture the full Image configuration.\nThe image configuration is just another layer from the registries perspective, so we can make this API call to get the Image configuration, from the digest we capture for the config from the image manifest.\nGET /v2/\u0026lt;name\u0026gt;/blobs/\u0026lt;digest\u0026gt; https://distribution.github.io/distribution/spec/api/#pulling-a-layer Now, I have a mechanism to gather the metadata of the container without actually pulling the container image file system layers.\nWe use AWS ECR as our Container Registry provider, at work, and AWS ECR supports the Docker Registry HTTP API.\nhttps://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html#registry_auth_http Now it\u0026rsquo;s as simple as making these calls and capturing the details during OS build.\n2. ecrimagemetadataextractor To do so I setup a very simple python CLI tool that I can invoke adhoc or within my CodeBuild project running the OS build job.\nhttps://github.com/Vasu77df/ecrimagemetadataextractor/tree/main You can get the wheel of this package from my release\nhttps://github.com/Vasu77df/ecrimagemetadataextractor/releases Or you can use the uv package manager to install, to learn more about uv checkout my post here\nWith this simple CLI tool I can run this before my actually task , and not really affect the task\u0026rsquo;s run time in a meaningful manner\nUsage:\nusage: ecrimagemetadataextractor [-h] [-v] -u IMAGE_URI [-r REGION] {get_manifest,get_digest_metadata} Simple CLI tool to extract the image manifest from private AWS ECR hosted container images positional arguments: {get_manifest,get_digest_metadata} Actions possible from the CLI, get_manifest: returns json manifest of container image, get_digest_metadata: returns json of first digest\u0026#39;s manifest options: -h, --help show this help message and exit -v, --verbose Verbose logging (default: False) -u IMAGE_URI, --image-uri IMAGE_URI uri of the container image in your registry example: 772738948692.dkr.ecr.us- east-1.amazonaws.com/os_build_env:latest (default: None) -r REGION, --region REGION aws region to use (default: None) Get Image Manifest\nCommand:\necrimagemetadataextractor get_manifest --image-uri 772738948692.dkr.ecr.us-east-1.amazonaws.com/os_build_env:latest --region us-east-1 | jq . Output:\n{ \u0026#34;schemaVersion\u0026#34;: 2, \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.distribution.manifest.v2+json\u0026#34;, \u0026#34;config\u0026#34;: { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.container.image.v1+json\u0026#34;, \u0026#34;size\u0026#34;: 4552, \u0026#34;digest\u0026#34;: \u0026#34;sha256:8a4a4313bd7cff420330aaf52e49da247b4c6107f44211d92db576e49b3f9659\u0026#34; }, \u0026#34;layers\u0026#34;: [ { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.image.rootfs.diff.tar.gzip\u0026#34;, \u0026#34;size\u0026#34;: 30610919, \u0026#34;digest\u0026#34;: \u0026#34;sha256:802008e7f7617aa11266de164e757a6c8d7bb57ed4c972cf7e9f519dd0a21708\u0026#34; }, { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.image.rootfs.diff.tar.gzip\u0026#34;, \u0026#34;size\u0026#34;: 238910491, \u0026#34;digest\u0026#34;: \u0026#34;sha256:6feaffcff6b9af7f741dfbc129b29e460ec4a1030c3c7684bc8ba15434982fcf\u0026#34; }, { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.image.rootfs.diff.tar.gzip\u0026#34;, \u0026#34;size\u0026#34;: 237477613, \u0026#34;digest\u0026#34;: \u0026#34;sha256:6dbf3486e2803e0e2d67068727038e05da509de9335ca7cdc7664b7833da48ad\u0026#34; }, { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.image.rootfs.diff.tar.gzip\u0026#34;, \u0026#34;size\u0026#34;: 67232420, \u0026#34;digest\u0026#34;: \u0026#34;sha256:e725b91ef7f278dd32339da93308782ab1af918204cf4d27082b374a089b159c\u0026#34; }, { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.image.rootfs.diff.tar.gzip\u0026#34;, \u0026#34;size\u0026#34;: 780, \u0026#34;digest\u0026#34;: \u0026#34;sha256:b3f5b9a15490c9c075140c809857f4724263b6637a9466ee8708879cee5331e9\u0026#34; }, { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.image.rootfs.diff.tar.gzip\u0026#34;, \u0026#34;size\u0026#34;: 28832838, \u0026#34;digest\u0026#34;: \u0026#34;sha256:f0eda69f6707fb23e5631437de1204664aad856f581270ba45ab5dadd0719e01\u0026#34; }, { \u0026#34;mediaType\u0026#34;: \u0026#34;application/vnd.docker.image.rootfs.diff.tar.gzip\u0026#34;, \u0026#34;size\u0026#34;: 6235879, \u0026#34;digest\u0026#34;: \u0026#34;sha256:c0cf9d8d47897c5a800086b3d4d80e28cc1653046b6795bccab4a9351025530a\u0026#34; } ] } Get image Configuration Metadata\nCommand:\necrimagemetadataextractor get_digest_metadata --image-uri 772738948692.dkr.ecr.us-east-1.amazonaws.com/os_build_env:latest --region us-east-1 | jq . Output:\n{ \u0026#34;architecture\u0026#34;: \u0026#34;amd64\u0026#34;, \u0026#34;config\u0026#34;: { \u0026#34;Env\u0026#34;: [ \u0026#34;PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\u0026#34; ], \u0026#34;Cmd\u0026#34;: [ \u0026#34;/bin/bash\u0026#34; ], \u0026#34;Labels\u0026#34;: { \u0026#34;maintainer\u0026#34;: \u0026#34;@vasuper\u0026#34;, \u0026#34;org.opencontainers.image.ref.name\u0026#34;: \u0026#34;ubuntu\u0026#34;, \u0026#34;org.opencontainers.image.version\u0026#34;: \u0026#34;24.04\u0026#34; } }, \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:35:38.524849246-05:00\u0026#34;, \u0026#34;history\u0026#34;: [ { \u0026#34;created\u0026#34;: \u0026#34;2024-10-11T03:48:01.529767151Z\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;/bin/sh -c #(nop) ARG RELEASE\u0026#34;, \u0026#34;empty_layer\u0026#34;: true }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-11T03:48:01.571862048Z\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;/bin/sh -c #(nop) ARG LAUNCHPAD_BUILD_ARCH\u0026#34;, \u0026#34;empty_layer\u0026#34;: true }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-11T03:48:01.607507065Z\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;/bin/sh -c #(nop) LABEL org.opencontainers.image.ref.name=ubuntu\u0026#34;, \u0026#34;empty_layer\u0026#34;: true }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-11T03:48:01.642491381Z\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;/bin/sh -c #(nop) LABEL org.opencontainers.image.version=24.04\u0026#34;, \u0026#34;empty_layer\u0026#34;: true }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-11T03:48:03.777394067Z\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;/bin/sh -c #(nop) ADD file:34dc4f3ab7a694ecde47ff7a610be18591834c45f1d7251813267798412604e5 in / \u0026#34; }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-11T03:48:04.086892655Z\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;/bin/sh -c #(nop) CMD [\\\u0026#34;/bin/bash\\\u0026#34;]\u0026#34;, \u0026#34;empty_layer\u0026#34;: true }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:33:50.683844544-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;LABEL maintainer=@vasuper\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34;, \u0026#34;empty_layer\u0026#34;: true }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:33:50.683844544-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;ARG DEBIAN_FRONTEND=noninteractive\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34;, \u0026#34;empty_layer\u0026#34;: true }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:33:50.683844544-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;RUN |1 DEBIAN_FRONTEND=noninteractive /bin/sh -c apt update -y \u0026amp;\u0026amp; apt upgrade -y \u0026amp;\u0026amp; apt install -y aptly bubblewrap ca-certificates cpio curl debian-archive-keyring dosfstools e2fsprogs git kmod procps python3-cryptography python3-pip python3-setuptools python3-venv python3-wheel rauc sbsigntool squashfs-tools systemd systemd-boot systemd-repart systemd-ukify systemd-container mtools tpm2-tools ubuntu-keyring unzip zstd # buildkit\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34; }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:34:17.339702189-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;RUN |1 DEBIAN_FRONTEND=noninteractive /bin/sh -c apt install -y libc6 groff less \u0026amp;\u0026amp; curl \\\u0026#34;https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip\\\u0026#34; -o \\\u0026#34;awscliv2.zip\\\u0026#34; \u0026amp;\u0026amp; ls -alh \u0026amp;\u0026amp; unzip ./awscliv2.zip \u0026amp;\u0026amp; ./aws/install # buildkit\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34; }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:34:25.791326946-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;RUN |1 DEBIAN_FRONTEND=noninteractive /bin/sh -c curl -o /tmp/amazon-ssm-agent.deb https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/debian_amd64/amazon-ssm-agent.deb \u0026amp;\u0026amp; dpkg -i /tmp/amazon-ssm-agent.deb \u0026amp;\u0026amp; curl -o /etc/amazon/ssm/amazon-ssm-agent.json https://raw.githubusercontent.com/aws/aws-codebuild-docker-images/master/ubuntu/standard/5.0/amazon-ssm-agent.json # buildkit\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34; }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:34:26.437300051-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;RUN |1 DEBIAN_FRONTEND=noninteractive /bin/sh -c rm --force --recursive /var/lib/apt/lists/* \u0026amp;\u0026amp; rm --force --recursive /usr/share/doc \u0026amp;\u0026amp; rm --force --recursive /usr/share/man \u0026amp;\u0026amp; apt clean # buildkit\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34; }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:34:32.860798461-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;RUN |1 DEBIAN_FRONTEND=noninteractive /bin/sh -c rm --force /usr/lib/python3.12/EXTERNALLY-MANAGED \u0026amp;\u0026amp; apt-get update \u0026amp;\u0026amp; apt-get install -y python3-pip \u0026amp;\u0026amp; pip install pefile # buildkit\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34; }, { \u0026#34;created\u0026#34;: \u0026#34;2024-10-13T10:35:38.524849246-05:00\u0026#34;, \u0026#34;created_by\u0026#34;: \u0026#34;RUN |1 DEBIAN_FRONTEND=noninteractive /bin/sh -c python3 -m venv mkosivenv \u0026amp;\u0026amp; mkosivenv/bin/pip install git+https://github.com/systemd/mkosi.git@v24.3 \u0026amp;\u0026amp; mkosivenv/bin/mkosi --version # buildkit\u0026#34;, \u0026#34;comment\u0026#34;: \u0026#34;buildkit.dockerfile.v0\u0026#34; } ], \u0026#34;os\u0026#34;: \u0026#34;linux\u0026#34;, \u0026#34;rootfs\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;layers\u0026#34;, \u0026#34;diff_ids\u0026#34;: [ \u0026#34;sha256:a46a5fb872b554648d9d0262f302b2c1ded46eeb1ef4dc727ecc5274605937af\u0026#34;, \u0026#34;sha256:99423e0f187f5cdb62e47051984634a6d11e157d3127ae506562857f80493928\u0026#34;, \u0026#34;sha256:aea99ec45e93114c710f03e635dfafd7320f429740cd5087822bc98365882429\u0026#34;, \u0026#34;sha256:fc1645534d5bec025b2a6fc13483396cd20184edcfe06580cc86579d0b2067cd\u0026#34;, \u0026#34;sha256:048f3fe585e6ebf6a57286b71bed90acc19fe8fef1410e898ccde7a42735fba8\u0026#34;, \u0026#34;sha256:47ea983fc16a48b103cf6faf2bc97ecca8463052e570651c5690c1dbfab0539c\u0026#34;, \u0026#34;sha256:76a0f1399709e7d3f747cd0380359338142de988544f5dea6298976afca4333b\u0026#34; ] } } ecrimagemetadataextractor currently only supports private ECR registries, and uses the IAM credentials in the environment it\u0026rsquo;s running and Basic Auth to gain access to the ECR registry provided.\nFor AWS CodeBuild run tasks it picks the image from CODEBUILD_BUILD_IMAGE environment variable present in every Containerized Codebuild task.\nMake sure you have the necessary ECR IAM credentials to pull layers and get images from your ECR.\nIn the future I might make this tool generic so that it can work with any public or private container registries, feel free to put in a pull request or fork this, if you are interested in extending this.\nHopefully you learnt about the first few layers of the onions behind a docker pull command and also how we could leverage some of the specifications, to get details of a container without needing to pull the entire image down or requiring the docker daemon or any other client.\n","permalink":"https://vasuper.vasu3797.workers.dev/posts/container_registry_api/","summary":"\u003cp\u003eIf you have ever run \u003ccode\u003edocker pull\u003c/code\u003e or pulled a container using the favorite container runtime tool of your choice, you\u0026rsquo;ve probably interacted with a container registry.\u003c/p\u003e\n\u003cp\u003eA container registry does not only provide mechanisms to store and distribute container images, it also exposes APIs that clients can use to perform other actions like fetching the container\u0026rsquo;s metadata without pulling the entire container.\u003c/p\u003e\n\u003cp\u003eAt work, I had a problem to solve, which is when I thought why not leverage these APIs to solve my problem. In this post we\u0026rsquo;ll see how I leveraged the Container Registry APIs for my benefit and possibly how you the reader can use it as well.\u003c/p\u003e","title":"Leveraging Container Registry APIs for your benefit."},{"content":"1. Motivation I wanted to put up a small snippet of code I did at work as a public github project, as it seemed like a common usage pattern that other folks can use of the shelf or copy from.\nWithin Amazon, we have our own python project management and dependency management tooling that automates away a lot of the boilerplate setup in defining and distributing a Python project.\nThey abstract away the setup and usage for tools and configurations like pyproject.toml, venv, mypy, black, pytest, etc. They essentially run these tools under the hood, but we as developers don\u0026rsquo;t need to care about setting these up every time we start a new project.\nThis also helps in standardizing linting rules, static type checker strictness, flake8 checks, etc, across a team, or organization.\nI wanted to test out some external alternatives, that perform a similar functions. My goal, write less boilerplate configuration, and only care about the code. I also need a easy way to manage my dependencies and, run my linting and, testing tools.\n2. The Tools I was suprised, there are a lot of tools to choose from. Here\u0026rsquo;s a list of some tools I came across and checked out.\nPeotry PDM Hatch Rye uv just the usual pip I ended up choosing uv, for the following reasons.\nas the documentation seemed well structured enough that it was easy to follow. It has a straight forward pip interface, if I am not really inclined to learn any uv specifics. It covered a lot of features like managing python versions a script executor project manager init, to automate some boilerplate setup of a project. add and remove dependencies. build projects for distributions and run projects in an isolated fashion It let\u0026rsquo;s you install and run tools like black and ruff(the makers of uv also made ruff). It has some utility functions to manage caches. uv advertises to be 10 to 100x faster than pip, and in my experience just downloading and adding dependencies really has been a lot faster than pip. My analysis was not purely a technical one, but was driven more on what I thought was easy to use. Another partial driver was the media hype when the tool was portrayed as an pip replacement. I wanted to check out what the hype is.\n3. Installing uv The uv project has got a standalone installer linked below, it\u0026rsquo;s as simple of running it to install uv.\nIt\u0026rsquo;s, always recommended to download and review scripts downloaded from the internet before running them. Maybe skip the pipe and add a -o to your curl and glance quickly if everything is dandy.\nhttps://docs.astral.sh/uv/getting-started/installation/ 4. Setting up a project I am working on building a CLI tool, and I want to setup my project as an application that can be packaged, and potentially just pipx installed (or maybe uv installed).\nTo init such a project run:\nuv init --app --package example You will see a directory of your project setup with this file structure, after running this command.\n❯ tree example example ├── pyproject.toml ├── README.md └── src └── example └── __init__.py 2 directories, 3 files 4. Setting up a virtual environment To start a venv for your project you can run this command:\nuv venv Once run you\u0026rsquo;ll see an output like so:\n❯ uv venv Using CPython 3.10.12 interpreter at: /usr/bin/python3 Creating virtual environment at: .venv Activate with: source .venv/bin/activate 5. Adding dependencies to your project Now let\u0026rsquo;s add dependencies to the project.\nFor my project I have two deps boto3 and requests\nYou can add dependencies with this command:\nuv add boto3 requests When you run this command uv will pull these dependencies into your project and update the pyproject.toml .\nuv aggressively caches packages across projects. If the dependencies have already been cached from another project, it will pull that into the current project and not re-download the dependencies.\n❯ uv add boto3 requests Resolved 12 packages in 364ms Built example @ file:///home/vasuper/personal_dev/example Prepared 5 packages in 371ms Installed 12 packages in 111ms + boto3==1.35.41 + botocore==1.35.41 + certifi==2024.8.30 + charset-normalizer==3.4.0 + example==0.1.0 (from file:///home/vasuper/personal_dev/example) + idna==3.10 + jmespath==1.0.1 + python-dateutil==2.9.0.post0 + requests==2.32.3 + s3transfer==0.10.3 + six==1.16.0 + urllib3==2.2.3 + ❯ cat pyproject.toml [project] name = \u0026#34;example\u0026#34; version = \u0026#34;0.1.0\u0026#34; description = \u0026#34;Add your description here\u0026#34; readme = \u0026#34;README.md\u0026#34; authors = [ { name = \u0026#34;Vasudevan Perumal\u0026#34;, email = \u0026#34;vasu3797@gmail.com\u0026#34; } ] requires-python = \u0026#34;\u0026gt;=3.10\u0026#34; dependencies = [ \u0026#34;boto3\u0026gt;=1.35.41\u0026#34;, \u0026#34;requests\u0026gt;=2.32.3\u0026#34;, ] [project.scripts] example = \u0026#34;example:main\u0026#34; [build-system] requires = [\u0026#34;hatchling\u0026#34;] build-backend = \u0026#34;hatchling.build\u0026#34; 6. Running your app or any command uv has an run command one can use to run a project or just any adhoc script. UV manages the dependencies for you without manually managing environments.\nuv run example ❯ uv run example Hello from example! You can also run any command with uv run within your project context. For example you can run the pytest against your project with:\nuv run pytest 7. Freezing an environment uv creates a uv.lock file that acts as an cross platform lock file, which contains the exact resolved versions that are installed in the project environment. This can be committed source control to provide consistent and reproducible installation on different machines.\nYou can also lock the dependencies declared in pyproject.toml into a requirements.txt with this command:\nuv pip compile pyproject.toml -o requirements.txt You can then use this requirements in your environment\nuv pip sync requirements.txt 8. Build the project for distribution You can build your project using uv by running\nuv build This will build the project into a distributable wheel or tarball.\nuv build Building source distribution... Building wheel from source distribution... Successfully built dist/example-0.1.0.tar.gz and dist/example-0.1.0-py3-none-any.whl 9. Publishing the project to a PyPI registry uv also supports publishing the built project to a pypi registry.\nuv publish You can setup a PyPI token with --token or UV_PUBLISH_TOKEN, or set a username with --username or UV_PUBLISH_USERNAME and password with --password or UV_PUBLISH_PASSWORD.\n10. The End uv is a really fast python management tool, and it\u0026rsquo;s user experience is pretty good. I would recommend it for folks looking for a tool to fit a similar need. In the next post I\u0026rsquo;ll share what I built and managed with uv.\nReferences https://astral.sh/blog/uv-unified-python-packaging https://docs.astral.sh/uv/ ","permalink":"https://vasuper.vasu3797.workers.dev/posts/using_uv/","summary":"\u003ch2 id=\"1-motivation\"\u003e1. Motivation\u003c/h2\u003e\n\u003cp\u003eI wanted to put up a small snippet of code I did at work as a public github project, as it seemed like a common usage pattern that other folks can use of the shelf or copy from.\u003c/p\u003e\n\u003cp\u003eWithin Amazon, we have our own python project management and dependency management tooling that automates away a lot of the boilerplate setup in defining and distributing a Python project.\u003c/p\u003e\n\u003cp\u003eThey abstract away the setup and usage for tools and configurations like pyproject.toml, venv, mypy, black, pytest, etc. They essentially run these tools under the hood, but we as developers don\u0026rsquo;t need to care about setting these up every time we start a new project.\u003c/p\u003e","title":"Using UV to manage a python project"},{"content":" These are my notes taken from learning about TPMs from this source material: https://courses.cs.vt.edu/cs5204/fall10-kafura-BB/Papers/TPM/Intro-TPM-2.pdf\nThese are just excerpts and notes, that I have captured for my recollection and understanding, from the chapter\nA common design goal across this diversity however, is the provision of some degree of secure processing, implemented in secure hardware.\nThe Trusted Platform Module, or TPM, is similar to a smart card device in that it is a small footprint low cost security module typically implemented as a tamper resistant integrated circuit (IC). The TPM however, has been specifically designed to be a building block for trusted computing\nOne major difference is that the TPM is considered to be a fixed token bound to a specific platform, whereas a smart card is a portable token traditionally associated with a specific user across multiple systems\nThis is not to say that the two technologies are mutually exclusive, but rather that they may be considered as complementary.\nTrusted Platforms In order to understand the TPM design requirements, it is first necessary to understand what the desirable features of a trusted platform are. To do this, a definition is required as to exactly what is meant by the term “trusted platform”.\nTCG defines trust to be “the expectation that a device will behave in a particular manner for a specific pur-pose\nProviding assurance of expected behaviour does not in itself provide any security. To achieve that, the entity relying on this assurance still has to ascertain that the “expected behavior” is indeed secure.\nDefinitions of a trusted platform:\nA Trusted Platform, is a computing platform that has a trusted component, probably in the form of built-in hardware, which it uses to create a foundation of trust for software processes.\nA Trusted Platform(TP) is defined as a computing platform that has a trusted component which is used to create a foundation of trust for software processes.\nThere is subtle distinction between trusted component, such as the TPM, and a trustworthy component.\nThe proper definition is that a trusted system or component is one whose failure can break the security policy, while a trustworthy system or component is one that won\u0026rsquo;t fail. Fundamental Features of a Trusted Platform. The minimum set of features that a trusted platform should have are: protected capabilities; integrity measurement; and integrity reporting.\nProviding support for these features leads to the definition of the security requirement of the TPM.\nProtected capabilities The concept of protected capabilities is used to \u0026ldquo;distinguish platform capabilities that must be trustworthy\u0026rdquo;.\nThese trustworthy protected capabilities are abilities to execute a command or set of commands on the TPM which access shielded locations where it is safe to operate on sensitive data. Examples of protected capabilities in the TPM include protection and reporting of integrity measurements (described below); and storage and management of cryptographic keys.\nIntegrity measurement and storage. Any trusted platform will need some means of measuring how it\u0026rsquo;s configured and what processes are running on that platform.\nAs a lot of software is executed on a system implicitly and not explicitly by the user it is hard to determine the legitimacy of process. For a platform to be trusted, that it has some means of measuring integrity of the process it is running This measurement should result in some form of integrity metric. Which Pearsons defines as \u0026ldquo;a condensed value of integrity measurements\u0026rdquo;.\nThe integrity metric can then be compared with acceptable values for a trusted store and value somewhere for later use. Of course such data would be held in secure storage locations such as would be provided by the protected capabilities described above.\nMeasuring platform integrity in this manner has to be a starting point. It is acceptable for a app to perform integrity measurements provided that the application itself is trustworthy. Even if the operating system can verify the integrity of the application, the integrity of the operating systemc has to be verified.\nUltimately there will be a some integrity measurement process that exists that cannot itself be verified. This process is known as the Root of Trust for measurements, or RTM. This is the starting point in the chain of integrity measurements, and ideally this process should run on tamper proof hardware, with execution code being stored in secure storage.\nIntegrity reporting The third requirement for a trusted platform is that it should be able to report its configuration to a challenger who requires this information in order to decide how much trust to place in the platform.\nPlatform should have means to report it\u0026rsquo;s integrity metrics, and to vouch for the accuracy of this information.\nThe challenger requires the a evidence to reply on to make trust decisions.\nThe implication here is that the integrity metrics can be signed by the trusted platform and that the challenger has a certificate that can be used to verify the signature.\nAdditional Features Confidentiality and integrity protection Secure storage Process isolation Confidentiality and integrity protection A trusted platform should provide both confidentiality and integrity protection to any data as required by the user. This should include but user data, and application code. These services should be available to protect data while it is being stored and during the execution of any process.\nSecure Storage The provision of confidentiality requires that a trusted platform is able to encrypt any user data for secure storage. Access to the data is controlled by a securely stored cryptographic key. Similar to encryption is the concept of sealing. In this case access to the data is controlled by platform state, the use of a cryptographic key is optional. This means that data can be sealed to a set of integrity metrics that reflect the platform state. The data will then only be accessible when the platform is in the desired configuration. The sealing mechanism can be used, for example, to ensure that no rogue applications are running before access is granted to sensitive data.\nProcess isolation The secure storage and integrity checking will protect data during storage. To protect data during execution the provision of process isolation is necessary.\nThe concept here is to provide an isolation kernel between the hardware and any operating system. This isolation kernel is used to create and manage multiple secure compartments that can exist in parallel, on the same machine Each can run its own OS and application in isolation from any other processes that are executing in parallel. In this way, app and data can be protected during execution.\nIsolation kernel greatly simplifies, the validation of acceptable integrity metrics: by isolating processes, the set of acceptable application platform configuration can be reduced to one operating system and one application only.\nTPM Features TPM Components TCG defines standards of TPM functionality, leaving it open for developers to impolement this functionality as they wish, either in hardware or software.\nTPM Standards do not specify the communications, interfaces or bus architecture, leaving these decisions to be made by the developer.\nI/O Block. The TCG do, however, specify an interface serialization transformation that allows data to be transported over virtually any bus or interconnect. This is one of the functionality of the I/O Block. This block manages information flow between the components, and between the TPM and the external bus. The control access rights to access TPM components are determined by flags maintained by the opt-in block.\nNon-volatile storage TPM has some NV memory to long term keys. two long term keys are stored in the NV memory. The first of these is the Endorsement Key(EK). the second is the Storage root Key(SRK), which forms the basis of a key hierarchy that manages secure storage.\nThe TPM also uses NV mem to store owner authorization data. This authorization data, in effect, is the owner\u0026rsquo;s password, not set by manufacturer, butt the user during the process of taking ownership of the TPM.\nEndorsement Keys It is a fundamental component of the TPM. For the TPM to operate, it must have an endorsement key pair, of which the private key is embedded in the TPM and never leaves it. The public EK is contained in a cert and is only used in a limited number of procedures.\nThe reason for limiting the use of the EK Cert is because the EK is unique to each TPM and consequently may be used to identify the device, and by extension the platform.\nTherefore, to protect user privacy when interacting with other entities, the use of the EK is restricted and internally generated aliases, the Attestation Identity Keys, or AIKs, are used for routine transactions.\nTPM manufacturers will provide the endorsement key pair and store this in tam- per resistant non-volatile memory before shipping the TPM. A certificate, or endorsement credential, can then be created which contains the public EK and information about the security properties of the TPM\nThis endorsement credential should be signed by a certification authority, known as the TPME or Trusted Platform Module Entity, who can attest to the fact that the key contained in the certificate is a public EK whose corresponding private EK is stored in a TPM that conforms to the TCG standards. This TPME may be a third party or, if authorised to do so, it may be the manufacturer themselves.\nSome organisations who wish to use the TPM may, for security reasons, pre- fer to use their own endorsement keys. To accommodate this, the standards allow the EK to be deleted and re-installed by the user. Of course if the user generated endorsement credential is not signed by a TPME, its use may be limited.\nThe purpose of the endorsement credential is to prove that the corresponding private EK is stored in a genuine TPM. So, in keeping with policy to control exposure of the public EK, the private EK is never used to generate signatures. Thus, the public EK is never required to verify signatures so it does not have to be widely distributed. The public EK is only used for encrypting data sent to the TPM during the process of taking ownership and the process of creating AIK certificates. These processes are described in sections 7.3.12 and 7.4.4. Encrypting data with the public EK ensures that the plaintext can only be recovered by the particular TPM identified in the endorsement credential.\nAttestation Identity Keys Both the EK and Storage Root key(SRK), never leave the device. The third type of key, the Attestation Identity Key(AIK), may also be stored within the TPM. This key may be regarded as an alias for the endorsement key.\nEach TPM can support many AIKs, thus the user can have may unlinkable keys that can be used to maintain anonymity between different service providers who require proof of identity. These AIKS must, therefore, be persistent. They could stored in the TPM NV mem, but for practical reasons the standards recommend keeping the AIK keys in secure external storage.\nThe TPM however must provide a volatile storage area where one or more AIK keys can be loaded when in use.\nPlatform Configuration Register. PCRs are unique feature of TPMs used to store integrity metrics. The metrics stored in these registers measure the integrity of any code, from BIOS to application, typically before the code is executed.\nPCR may be implemented n volatile or NV storage, however the registers must be reset, whenever the system loses power or re-starts. If not old metrics may remain for new configurations after platform re-boot.\nTPM must have at least 16 Platform Configuration Registers and that each register stores 20 bytes. Registers 0 to 7 are reserved for exclusive use by the TPM, the remaining registers are free for use by the operating system and any application.\nProgramme Code In common with smart cards, the TPM requires storage for the firmware that is used to initialise the device. If the programme code is stored permanently on the tamper proof TPM then it would be reasonable to assume that it is trustworthy. Thus there would be no need to check its integrity making this the obvious location to store the code that carries out the integrity checks on all other platform devices and code. That is to say, the programme code on the TPM is the obvious “root of trust” for integrity measurements described in section 7.2. The TCG refer to such a root of trust as the CRTM, or Core Root of Trust for Measurement. Although the TPM programme code is the obvious choice for the CRTM, implementation decisions often require the CRTM be located in other firmware such as the BIOS boot block. Regardless of where the CRTM resides it should be considered as a trusted component of the system since if it fails all security policy based on integrity measurements will be broken.\nExecution Engine. The TPM has an execution engine which runs the programme code described above. The execution engine responds to external commands by se lecting the required programme code and executing it on the TPM.\nRandom Number Generator TPM includes a true random bit stream generator. Implementation of this is left to the developers so long the random source is used, rather than a deterministic method. Random bit streams are used to see a random number generator. The random numbers produced can be used to construct keys for symmetric cryptographic applications. The random numbers may also be used to provide nonces and, by mixing with user input, to increase the entropy in pass phrases.\nSHA-1 Engine The SHA-1 message digest engine is an implementation of the Secure Hash Algorithm [6] SHA-1. This algorithm hashes the input data and produces a 20-byte digest. It also forms the basis of an HMAC [7, 8] (Hash Based Message Authentication Code) engine, and is used in a number of cryptographic procedures carried out by the TPM, for example: in the computation of digital signatures and for creating key objects where a hash of the key may be required as part of an integrity protection mechanism.\nRSA Key Generation Generating keys suitable for use with the RSA algorithm [11] can be a computa-tionally intensive task and since such keys are widely used in the TPM, for signing and providing secure storage\nAll implementations of the TPM are required to support up to 2048 bit RSA, the Standard specifies that the TPM must be able to support upto 2048 bit modulus.\nRSA Engine Just as the generation of RSA keys is computationally complex, so is the execution of the algorithm itself. Therefore, the standards also require the TPM to have a ded- icated RSA engine used to execute the RSA algorithm. The RSA algorithm is used for signing, encryption, and decryption\nOpt-In The opt-in component and the concept of ownership, represents on of the biggest differences between smart cards and the TPM. Smart cards are usually owner byt the Issuer before the consumer recieves the device. TCG, is conscious of the perception that the TPM enabled platforms will somehow be controlled by large remote organisations, have been very careful to provide mechanisms to ensure that it is the user who takes ownership and configures the TPM.\nDuring the process of taking ownership, the TPM will make transitions through a number of states depending on the initial state in which the device was shipped. The state in which the TPM exists is determined by a number of persistent and volatile flags. Changing the state of these flags requires authorization by the TPM owner, if he exists, or demonstration of physical presence\nTpm operational States There are several mutually exclusive states in which the TPM can exist from disabled and deactivated, through to fully enabled and ready for an owner to take possession.\nFor a TPM to be owned by a user, it must have an endorsement key pair, and a secret owner authorisation data known by the owner. Once in the owned state, the owner of the TPM may perform all operations including operational state change. The TPM needs to have an owner and be enabled for all functions to be available\nTaking ownership When a user takes ownership of a TPM they establish a shared secret, referred to as owner authorization data, and insert this data into the secure storage on the TPM. This is in effect, the owner\u0026rsquo;s password. and being to demonstrate knowledge of this secret provides proof of ownership.\nThe process of taking ownership requires that the owner authorization data is protected from eavesdropping or theft by a malicious third party. This is one case where the endorsement cred is used.\nIt is the EK that establishes the secure channel to transmit the authorization data to a genuine TPM. So, during the process of taking ownership. So, during the process of taking ownership, the owner requests the endorsement credential and after verifying this credential, retrieves the public EK and uses this to encrypt the shared secret. Only the TPM identified in the endorsement credential has access to the provide EK, therefor the shared secret is only made available to the intended TPM.\nTPM Services The foundation for the provision of these services is the concept of a \u0026ldquo;root of trust\u0026rdquo; from which other services such as authenticated boot, secure storage, and attestation can be constructed.\nRoots of Trust In the trusted platform architecture by the TCG, there are three distinct roots of trust:\na Root of Trust for Measurement(RTM) a Root of Trust for Storage(RTS) a Root of Trust for Reporting(RTR) RTM must be trusted to generate integrity measurement of the processes that are running on the platform. This should ideally be a tamper proof component that boots very early in the boot process and therefor is able to measure all other components that are loaded after it. The TPM is an ideal candidate for a CRTM Core Root of trust measurement\nCRTM: The first piece of BIOS code that executes on the main processor during the boot process. On a system with a Trusted Platform Module the CRTM is implicitly trusted to bootstrap the process of building a measurement chain for subsequent attestation of other firmware and software that is executed on the computer system.\nRTS is the component that provides confidentiality and integrity protection. The RTS can then trusted to store either data, e.g PCR, or keys such as the SRK that allow data to be securely stored in external locations.\nFinally, the Root of Trust for Reporting is a trusted component that provides reports of any integrity measurements that may have been made - attesting to the platform configuration.\nBoot process TBB: Trusted Boot block\nNot illustrated in the diagram, the CRTM should measure the rest of the BIOS before loading it. Once the BIOS is loaded it takes control and measure the integrity of the OS loader as shown in step 1. Control then passes to OS loader in step 2, and the OS loader then measure the integrity of the operating system. This process continues until the applications are loaded and executed in steps 6.\nThe integrity measurement at each stage are made by creating a SHA-1 digest of the code to be loaded. This digest is stored in one of the PCR registers, which are initialized to zero. The new integrity metric however does not simply overwrite the old PCR value. The process of updating(or extending) the PCR value concatenates the 20 bytes of data already held in the PCR with the 20 bytes of the new data calculated These 40 bytes of data are then hashed again using the SHA-1 algo and the result written to the original PCR. In pseudo code: PCR \u0026lt;- has(PCR||hash(new code)). This way the PCR can store an unlimited number of measurements.\nIn order to interpret the value contained in the PCR, it is necessary to know the individual digests that have been added to it. These data are stored externally in what the TCG refer to as the Stored Measurement Log. Thus, if the data in the stored measurement log is known, and the PCR values are known, and trusted, then a challenger can verify the state of the platform.\nSecure Storage In Taking ownership section above the process of taking ownership resulted in the creation of a Storage Root Key, or SRK. This key is generated by the TPM and never leaves the device. It can only be accessed by demonstrating knowledge of a shared secret, in this case the SRK authorization data. This shared secret is similar to the owner authorization data and loaded into the TPM at the same time, during the process of taking ownership\nThe SRK forms the root of a key hierarchy as illustrated in figure 7.3 which has also been taken from the TCG Architecture Overview [16]. This key hierarchy allows data, or keys, to be encrypted such that they can only be decrypted by accessing the TPM\nThe TPM provides two mechanisms for secure storage: binding and sealing. The binding operation encrypts the data using a key that is managed by a particular TPM as described above. The sealing process adds to this by only allowing the deciphering process to proceed if the platform is in a specific configuration. This configuration is determined by data held in the PCR registers. Thus, when data is sealed, not only must the same platform be used to unseal the data, but that platform must be in a predetermined configuration before the data can be recovered.\nAttestation The AIK keys shown in the figure above are the Attestation Identity Keys. The purpose of the AIK is to provide users privacy when communicating with different sources. Although EK could be used to secure communication with different sources, since the EK is unique to the TPM, this could potentially allow the platform\u0026rsquo;s identity to be linked between each source it chose to communicate with.\nThe idea of the AIK is to provide a unique unlinkable identity for the TPM, for use with each different source. In each case the AIK acts as an alias for the EK. The AIK credential is a certificate containing the AIK public key which proves that the corresponding private key is bound to a genuine TPM. This proof is guaranteed by a signature on the credential created by a trusted third party known as a “privacy CA”.\nTo obtain an AIK a request is sent to the privacy CA together with the endorse- ment credential. This the second case where the public EK is exposed. The endorse- ment credential proves to the privacy CA that the request came from a genuine TPM. In response to the request, the privacy CA creates and signs the AIK credential, and encrypts this under the public EK contained in the endorsement credential. Thus the AIK is cryptographically bound to the TPM that contains the private EK. The users can create as many AIK keys as they wish and provided the privacy CA is trust- worthy, these keys will remain unlinkable and provide the privacy required when communicating with different sources. The private AIK key is managed by the TPM as illustrated in 7.3 and may be freely used to generate signatures. In particular, the private AIK can sign the con- tents of the PCR registers. This is how the TPM can be used to attest to the platform configuration. A relying party in possession of a public AIK can now challenge the trusted platform, and the attestation mechanism can be used to provide an integrity report describing the platform state.\n","permalink":"https://vasuper.vasu3797.workers.dev/posts/intro-to-tpm/","summary":"\u003chr\u003e\n\u003cp\u003eThese are my notes taken from learning about TPMs from this source material: \u003ca href=\"https://courses.cs.vt.edu/cs5204/fall10-kafura-BB/Papers/TPM/Intro-TPM-2.pdf\"\u003ehttps://courses.cs.vt.edu/cs5204/fall10-kafura-BB/Papers/TPM/Intro-TPM-2.pdf\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eThese are just excerpts and notes, that I have captured for my recollection and understanding, from the chapter\u003c/p\u003e\n\u003chr\u003e\n\u003cp\u003eA common design goal across this diversity however, is the provision of some degree of secure processing, implemented in secure hardware.\u003c/p\u003e\n\u003cp\u003eThe Trusted Platform Module, or TPM, is similar to a smart card device in that it is a small footprint low cost security module typically implemented as a tamper resistant integrated circuit (IC). The TPM however, has been specifically designed to be a building block for trusted computing\u003c/p\u003e","title":"My notes on the Trusted Platform Modules"},{"content":"Greengrass Concepts Basics GG is a edge runtime with a deployment and management mechanism for the egde runtime\nAWS IoT thing: A device or logic entity\nGreengrass core device: Device running greengrass core software\nGreengrass client device: A device that connects to and communicates with a greengrass core device over MQTT.\nclient device and core device could the same device or could be separated, with a sort of master slave relationship. client devices are meant for small footprint device that run embedded linux or RTOS maybe?\nGreengrass component: A software module that is deployed on a Greengrass core device. All software developed and deployed through Greengrass is a modeled as a component.\nRecipe: A JSON and YAML that describes the software module. Artifacts: The source code, binaries, or scripts Dependency: The relationship between components that enable you to enfore automatic updates or restarts of dependent components. Greengrass Core Software The set of al greengrass software that you install on a core device.\nNucleus: The required component that provides the minimum functionality of the aws iot greengrass core software. The nucleus manages deployment, orchestration, and lifecycle management of other components. Optional components: Configurable components Features of AWS IoT Greengrass Software distributions\nNucleus AWS provided components Greengrass development tools AWS IoT SDK for IPC library Stream manager SDK Cloud services\nAPI Console GG Core Software Features: Process data streams on local device and automatic export to AWS Cloud MQTT messaging between AWS IoT and components Support local publish and subcribe messaging between components Deploy and invoke components and lambda functions. Manage component lifecycle. Perform secure OTA updates of core software and custom components Provide secure, encrypted storage of local secrets and controlled access by components. Secure connections between devices and aws cloud with device auth My thoughts on Greengrass Components vs AWS IoT jobs At work, we are working on a remote OS update mechanism for our custom downstream ubuntu distro that updates atomically through a image-based update mechanism. The actual OS updates i.e downloading an update bundle and writing it to disk is managed the RAUC. These are my notes taken while evaluating whether to use Greengrass Components or IoT jobs to remotely trigger this agent, in hindsight\u0026rsquo;s it\u0026rsquo;s clear IoT jobs is the appropriate mechanism, but these notes depict how I got to that decision.\nAWS IoT jobs is the right tool to use for IFTTT type operations. AWS IoT Job is akin to ansible playbooks.\nA Greengrass Components is right tool to use a. To package your software, in it\u0026rsquo;s supported formats. b. To deploy said software. A Greengrass component is akin to docker-compose.\nIoT jobs is the right tool to use to run a task(OS upgrades are tasks), as it has in-built mechanisms of state management that we would have to re-invent with a Greengrass component. For now we expect state management of an OS upgrade as a simple 3 step process, but this state complexity might grow in the long run.\nIf we define, a Greengrass component in such a manner to run a task, we are not using Greengrass components for what it was intended to do. Which is alright, if we really have to pick one or the other. But as we have the ability to choose right tool for the job, we should use IoT jobs to run tasks.\nFrom CI/CD pipelines or orchestrator perspective, it may seem like it simplification if we use only one specific mechanism to run tasks on a device, but from device lifecycle perspective, IoT jobs and Greengrass components are completely different things with different purposes.\nIoT jobs will be recommended interface for customers to define and deploy one-off or even continuous tasks on the device. Greengrass components, will be recommended interface to deploy applications, and maintain it\u0026rsquo;s lifecycle. This is how I look at this, I understand both mechanisms can be altered to either of the operations, but if that mechanism increases in complexity, the author as to be very aware of what they are doing.\n","permalink":"https://vasuper.vasu3797.workers.dev/posts/greengrass_notes/","summary":"\u003ch1 id=\"greengrass-concepts-basics\"\u003eGreengrass Concepts Basics\u003c/h1\u003e\n\u003cp\u003eGG is a edge runtime with a deployment and management mechanism for the egde runtime\u003c/p\u003e\n\u003cp\u003e\u003cimg alt=\"gg_arc\" loading=\"lazy\" src=\"/gg_post.png\"\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAWS IoT thing\u003c/strong\u003e: A device or logic entity\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGreengrass core device\u003c/strong\u003e: Device running greengrass core software\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGreengrass client device\u003c/strong\u003e: A device that connects to and communicates with a greengrass core device over MQTT.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eclient device\u003c/em\u003e and \u003cem\u003ecore device\u003c/em\u003e could the same device or could be separated, with a sort of master slave relationship. client devices are meant for small footprint device that run embedded linux or RTOS maybe?\u003c/p\u003e","title":"My Notes while learning about Greengrass"},{"content":"Hello, My name is Vasudevan Perumal.\nI\u0026rsquo;m an computer systems generalist, with a some battle scars(aka experience) in Linux distribution engineering, and building systems to provision and manage devices at scale using cloud-native technologies. Right now, I build cloud-native endpoints for robotics systems at Amazon. What is a cloud-native endpoint you might ask?\nCloud-native endpoints are devices that can be deployed from anywhere. They receive their applications and configurations dynamically from the cloud, and can easily be reset or restored. These systems don\u0026rsquo;t generally require a direct connection to any on-premises resources for usability or management.\nTo know me better, and what I am about, quick, I think sharing some principles I follow is the best way. Here are some are these principles.\nAlways reason from first principles. Respect what came before. Have strong opinions, but loosely held. Be stubborn on the vision, flexible on the approach. Iterative improvement, than perfection. I really love tinkerering and, I never shy away from a problem stating, it\u0026rsquo;s not my expertise. I try my best to learn any domain, dive deep, and find an holistic answer. The way I do this, is through brute force probing the question, and learning histroically what has been the answer. History is a great informer, of the current problem you have to solve. I do this through search tool(google, the best AI model I have access to etc.), and reading as much documentation I can. With an informed understanding, I then reachout to team members and folks with experience to share my perspective and learn from their experience. Running through this loop a couple of times, as always yielded with a deeper understanding, and answers that are not only correct but when implemented are sustainable for the longer term.\nWhat am I doing now as 04/06/2025? I have been diving deeper in my quest to understand compute systems. This is a field I fell into, I started out to become a network engineer.\nI have been learning Rust, and system programming techniques, undertanding Linux internals better, like sockets. Learning async programming in python, and Rust, this topic I have not dabbled with professionally yet.\nOn the side, I am also working on Project Lamper, as a method to learn new agentic AI technologies. The problem I want to solve is to make the Sys admin job as accessible to non-technical folks as possible. Making the average person, self-sufficient to fix their IT problems on their own. Through an architecture of running SLM(small language models) like Gemma3:1b locally, and the combination of RAG for context augmenation, fine-tuning for context awareness of your system, and finally function calling and local inference. My aim is to take a open ended my computer freezes sometimes, and completely local mechanism for anyone to understand why their computer is behaving in a certain way. The practicality of this solution is yet to tested, hopefully quantization techniques improve exponentially over the years.\n","permalink":"https://vasuper.vasu3797.workers.dev/about/","summary":"\u003cp\u003eHello,\nMy name is Vasudevan Perumal.\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;m an computer systems generalist, with a some battle scars(aka experience) in Linux distribution engineering, and building systems to provision and manage devices at scale using cloud-native technologies. Right now, I build cloud-native endpoints for robotics systems at Amazon. What is a cloud-native endpoint you might ask?\u003c/p\u003e\n\u003cp\u003eCloud-native endpoints are devices that can be deployed from anywhere. They receive their applications and configurations dynamically from the cloud, and can easily be reset or restored. These systems don\u0026rsquo;t generally require a direct connection to any on-premises resources for usability or management.\u003c/p\u003e","title":"About me"},{"content":"In this guide, we\u0026rsquo;ll be going over the steps involved to build a LiveCD Debian distribution using debootstrap with both legacy BIOS and UEFI boot support using GRUB2. Why build your own distribution, you ask? There are scenarios and use cases that necessitate packages, certificates or other artifacts are already installed in the OS Image that\u0026rsquo;s distributed in the machines that are shipped, this guide will go over the steps involved to build that image.\nPrerequisites Let\u0026rsquo;s begin by installing the build tools.\nsudo apt install \\ debootstrap \\ squashfs-tools \\ xorriso \\ isolinux \\ syslinux-efi \\ grub-pc-bin \\ grub-efi-amd64-bin \\ grub-efi-ia32-bin \\ mtools \\ dosfstools \\ parted \\ debian-archive-keyring Create a directory where we will store all of the files we create throughout this guide.\nmkdir -p $HOME/LIVE_BOOT Bootstrap and Configure Debian Set up the base Debian environment. I\u0026rsquo;m using bullseye for my distribution and amd64 for the architecture. Consult the list of debian mirrors. Please change the URL in this command if there is a mirror that is closer to you.\nsudo debootstrap \\ --arch=amd64 \\ --variant=minbase \\ bullseye \\ $HOME/LIVE_BOOT/edit \\ http://ftp.us.debian.org/debian/ Copy resolver settings We\u0026rsquo;re copying the DNS resolver settings from the host to the root filesystem of the image we are building so that we can resolve DNS request while within the chroot that we\u0026rsquo;ll be pivoting into in the next step.\nsudo mkdir -p $HOME/LIVE_BOOT/edit/run/systemd/resolve sudo cp -f /etc/resolv.conf $HOME/LIVE_BOOT/edit/run/systemd/resolve/stub-resolv.conf Prepare and chroot Chroot to the Debian environment we just bootstrapped and mounts directories linux expects to handles processes and block devices\nsudo mount -t proc /proc $HOME/LIVE_BOOT/edit/proc sudo mount -t sysfs /sys $HOME/LIVE_BOOT/edit/sys sudo mount -o bind /dev $HOME/LIVE_BOOT/edit/dev sudo mount -o bind /dev/pts $HOME/LIVE_BOOT/edit/dev/pts sudo chroot $HOME/LIVE_BOOT/edit /bin/bash export HOME=/root export LC_ALL=C [chroot] Set a custom hostname for your Debian environment.\necho \u0026#34;custom-debian-machine\u0026#34; \u0026gt; /etc/hostname [chroot] add hostname to hosts file and update the nameservers\ncat \u0026gt; /etc/hosts \u0026lt;\u0026lt; EOF 127.0.0.1 localhost 127.0.1.1 $(cat /etc/hostname) # The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback ff02::1 ip6-allnodes ff02::2 ip6-allrouters EOF cat \u0026lt;\u0026lt; EOF \u0026gt; /run/systemd/resolve/stub-resolv.conf nameserver 8.8.8.8 nameserver 1.1.1.1 EOF [chroot] Install a Linux kernel of your choosing. I chose the image linux-image-amd64. We also install live-boot which provides the necessary hooks for LiveCD system. We install systemd-sysv (or an equivalent) as it\u0026rsquo;s necessary for an init system. Let\u0026rsquo;s install some other packages we might need in our distribution. You can also add packages to this list to get installed as part of the image you are building, allowing you to further customize the image to include software you want to distribute with your distro.\napt update \u0026amp;\u0026amp; \\ apt install --no-install-recommends \\ linux-image-amd64 \\ live-boot \\ systemd-sysv \\ vim \\ less \\ dnsmasq \\ network-manager \\ net-tools \\ wireless-tools \\ curl \\ openssh-client \\ procps \\ wget \\ gnupg2 \\ sudo \\ ca-certificates \\ iputils-ping [chroot] Installing mosquitto This is a MQTT agent that facilitates a lot IoT based interactions with cloud-services or your local MQTT based IoT platform. This step can be ignores if you don\u0026rsquo;t have this requirement.\nwget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key apt-key add mosquitto-repo.gpg.key apt-get update apt-get install mosquitto apt-get install mosquitto-clients [chroot] Downloading the AmazonRootCA.pem Cert for various AWS IoT communications, can be ignored if not needed for your use case.\nwget https://www.amazontrust.com/repository/AmazonRootCA1.pem -P /etc/pki # check contents with. openssl x509 -in /etc/pki/AmazonRootCA1.pem -text -noout [chroot] Cleaning up\napt-get clean apt-get autoclean apt-get autoremove # remove bash history rm -rf /tmp/* ~/.bash_history [chroot] Adding a user and setting password for root** Make sure to change this and not use the default. You can generated salted passwords using this command\nopenssl passwd -6 -salt xyz password # password is password useradd -G sudo -m -U -s /bin/bash -p \\ $6$pass$75vTrf3kWE4ncL.vF1TixnLx7qs34pt3tlgm/6uXOhccJXsb9LrY.d3izKxAYnzlRvRwkolgdRlipd.eNrE8g. \\ user [chroot] Exit chroot and un-mount all special directories\nexit sudo umount $HOME/LIVE_BOOT/edit/proc sudo umount $HOME/LIVE_BOOT/edit/sys sudo umount $HOME/LIVE_BOOT/edit/dev/pts sudo umount $HOME/LIVE_BOOT/edit/dev Building the image with GRUB2 Create directories that will contain files for our live environment and scratch files.\nmkdir -p $HOME/LIVE_BOOT/{scratch,image/live} Compress the chroot environment into a Squash filesystem.\nsudo mksquashfs \\ $HOME/LIVE_BOOT/edit \\ $HOME/LIVE_BOOT/image/live/filesystem.squashfs \\ -e boot Copy the kernel and initramfs from inside the chroot to the live directory.\ncp $HOME/LIVE_BOOT/chroot/boot/vmlinuz-* \\ $HOME/LIVE_BOOT/image/vmlinuz \u0026amp;\u0026amp; \\ cp $HOME/LIVE_BOOT/chroot/boot/initrd.img-* \\ $HOME/LIVE_BOOT/image/initrd Create a menu configuration file for grub.\nThis config instructs Grub to use the search command to infer which device contains our live environment. This seems like the most portable solution considering the various ways we may write our live environment to bootable media.\ncat \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; \u0026gt;$HOME/LIVE_BOOT/scratch/grub.cfg insmod all_video search --set=root --file /ZTP_IMAGE set default=\u0026#34;0\u0026#34; set timeout=30 menuentry \u0026#34;Debian Live\u0026#34; { linux /vmlinuz boot=live quiet nomodeset initrd /initrd } EOF Create a special file in image named DEBIAN_IMAGE. This file will be used to help Grub figure out which device contains our live filesystem. This file name must be unique and must match the file name in our grub.cfg config.\ntouch $HOME/LIVE_BOOT/image/DEBIAN_IMAGE Adding BIOS and UEFI boot support Create a grub UEFI image.\ngrub-mkstandalone \\ --format=x86_64-efi \\ --output=$HOME/LIVE_BOOT/scratch/bootx64.efi \\ --locales=\u0026#34;\u0026#34; \\ --fonts=\u0026#34;\u0026#34; \\ \u0026#34;boot/grub/grub.cfg=$HOME/LIVE_BOOT/scratch/grub.cfg\u0026#34; Create a FAT16 UEFI boot disk image containing the EFI bootloader. Note the use of the mmd and mcopy commands to copy our UEFI boot loader named bootx64.efi\n(cd $HOME/LIVE_BOOT/scratch \u0026amp;\u0026amp; \\ dd if=/dev/zero of=efiboot.img bs=1M count=10 \u0026amp;\u0026amp; \\ sudo mkfs.vfat efiboot.img \u0026amp;\u0026amp; \\ mmd -i efiboot.img efi efi/boot \u0026amp;\u0026amp; \\ mcopy -i efiboot.img ./bootx64.efi ::efi/boot/ ) Create a grub BIOS image.\ngrub-mkstandalone \\ --format=i386-pc \\ --output=$HOME/LIVE_BOOT/scratch/core.img \\ --install-modules=\u0026#34;linux normal iso9660 biosdisk memdisk search tar ls\u0026#34; \\ --modules=\u0026#34;linux normal iso9660 biosdisk search\u0026#34; \\ --locales=\u0026#34;\u0026#34; \\ --fonts=\u0026#34;\u0026#34; \\ \u0026#34;boot/grub/grub.cfg=$HOME/LIVE_BOOT/scratch/grub.cfg\u0026#34; Use cat to combine a bootable Grub cdboot.img bootloader with our boot image.\ncat \\ /usr/lib/grub/i386-pc/cdboot.img \\ $HOME/LIVE_BOOT/scratch/core.img \\ \u0026gt; $HOME/LIVE_BOOT/scratch/bios.img Generate the ISO file.\nxorriso \\ -as mkisofs \\ -iso-level 3 \\ -full-iso9660-filenames \\ -volid \u0026#34;DEBIAN_IMAGE\u0026#34; \\ -eltorito-boot \\ boot/grub/bios.img \\ -no-emul-boot \\ -boot-load-size 4 \\ -boot-info-table \\ --eltorito-catalog boot/grub/boot.cat \\ --grub2-boot-info \\ --grub2-mbr /usr/lib/grub/i386-pc/boot_hybrid.img \\ -eltorito-alt-boot \\ -e EFI/efiboot.img \\ -no-emul-boot \\ -append_partition 2 0xef ${HOME}/LIVE_BOOT/scratch/efiboot.img \\ -output \u0026#34;${HOME}/LIVE_BOOT/custom-debain-distro.iso\u0026#34; \\ -graft-points \\ \u0026#34;${HOME}/LIVE_BOOT/image\u0026#34; \\ /boot/grub/bios.img=$HOME/LIVE_BOOT/scratch/bios.img \\ /EFI/efiboot.img=$HOME/LIVE_BOOT/scratch/efiboot.img Making the raw disk image We would need a raw disk image to provide to vendors that they dd as a lot of manufacturers expect the image to be in this format for imaging clients on the factory floor\nWe already have built an iso file. We can use that to create our raw disk image.\nCreate a new sparse disk image sudo dd if=/dev/zero of=custom-debian.img bs=1 count=0 seek=400M Partition the disk sudo parted -s custom-debian.img mklabel gpt sudo parted --script --align=optimal custom-debian.img mkpart ESP fat32 1MiB 100MiB sudo parted --script --align=optimal custom-debian.img mkpart ext4 100MiB 100% sudo parted --script custom-debian.img set 1 boot on # checkout the partitions sudo parted -s custom-debian.img print Check which loop device we can use sudo losetup --show -f custom-debian.img # example output #/dev/loop15 Create loop device sudo losetup -P /dev/loop15 custom-debian.img Write iso to loop device, which writes to raw disk sudo dd if=custom-debian-distro.iso of=/dev/loop15 Cleanup the loop device sudo losetup -d /dev/loop15 now you should have have a bootable raw disk image custom-debian.img , you can learn more about the disk running this command\nfdisk -l custom-debian.img Some Tips and Tricks if you are building this on an EC2 instance You need to run a sudo apt update \u0026amp;\u0026amp; apt upgrade before installing the build tools on first EC2 provisioning or as the package has not been setup: You need to provide your own resolv.conf as copying the resolv.conf from the host machine i.e the EC2 instance points to internal ec2 nameservers and wouldn’t work externally. A better instance type might help in speed of building the squashfs filesystem. The t2.micro takes about a solid 2mins. On an ec2 instance you cannot use the mkfs.vfat unless you are root, I updated the command with sudo prepended. A detail to consider. The debian AMI ec2 instance doesn\u0026rsquo;t have parted command installed. I\u0026rsquo;ve added parted to the prereqs build tools. References https://www.willhaley.com/blog/custom-debian-live-environment-grub-only/ (major thanks to Will Haley for this guide) ","permalink":"https://vasuper.vasu3797.workers.dev/posts/debian-image-debootstrap/","summary":"\u003cp\u003eIn this guide, we\u0026rsquo;ll be going over the steps involved to build a LiveCD Debian distribution using debootstrap with both legacy BIOS and UEFI boot support using GRUB2. Why build your own distribution, you ask?\nThere are scenarios and use cases that necessitate packages, certificates or other artifacts are already installed in the OS Image that\u0026rsquo;s distributed in the machines that are shipped, this guide will go over the steps involved to build that image.\u003c/p\u003e","title":"Build your own Customized Live Debian Distro using Debootstrap"},{"content":"This guide outlines the steps involved to build a linux distribution using Yocto for AMD Ryzen Embedded v1000 Series SoCs with AWS SDKs, services, and software from the meta-aws layer.\nPrerequisites: To follow the steps image build section, you will need:\nA Supported Linux Distribution: You should have a reasonably current Linux-based host system. You will have the best results with a recent release of Fedora, openSUSE, Debian, Ubuntu, or CentOS as these releases are frequently tested against the Yocto Project® and officially supported. I’ve tried building this with both Ubuntu 20.04 and Pop_OS! 22.04.\nRequired Packages for the Build Host:\nInstalling the required packages on a Debian based distribution(Ubuntu, Pop_OS!,etc): sudo apt install -y gawk wget git diffstat unzip texinfo gcc \\ build-essential chrpath socat cpio python3 python3-pip \\ python3-pexpect xz-utils debianutils iputils-ping python3-git \\ python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm \\ python3-subunit mesa-common-dev Other steps you might need to do:\nBecause the Yocto Project tools rely on the “python” command, you will likely need to alias python to python3. Edit your .bashrc or .zshrc file and add: alias python=python3 restart your terminal or run source ~/.bashrc || source ~/.zshrc run python --version and checkout if you are running the python3 version The Layers we’ll be using: Poky reference distribution layer: The poky reference distribution layer is the common base for all yocto base builds.\nYou will want your poky layer branch to match the branches of all other third-party layers you download. You can view the available poky release names here: https://wiki.yoctoproject.org/wiki/Releases. We’re going to have all layers be on the kirkstone branch.\nThe meta-openembedded Layer: This layer is going to help us to get features like networking , python, a filesystem, a desktop environment etc. below are the list of layers in meta-openembedded: ❯ git clone -b kirkstone git://git.openembedded.org/meta-openembedded ❯ cd meta-openembedded \u0026amp;\u0026amp; ll total 52K drwxr-xr-x 2 vasuper domain^users 4.0K Sep 27 08:27 contrib -rw-r--r-- 1 vasuper domain^users 1.1K Sep 27 08:27 COPYING.MIT drwxr-xr-x 6 vasuper domain^users 4.0K Sep 27 08:27 meta-filesystems drwxr-xr-x 10 vasuper domain^users 4.0K Sep 27 08:27 meta-gnome drwxr-xr-x 8 vasuper domain^users 4.0K Sep 27 08:27 meta-initramfs drwxr-xr-x 10 vasuper domain^users 4.0K Sep 27 08:27 meta-multimedia drwxr-xr-x 17 vasuper domain^users 4.0K Sep 27 08:27 meta-networking drwxr-xr-x 25 vasuper domain^users 4.0K Sep 27 08:27 meta-oe drwxr-xr-x 5 vasuper domain^users 4.0K Sep 27 08:27 meta-perl drwxr-xr-x 9 vasuper domain^users 4.0K Sep 27 08:27 meta-python drwxr-xr-x 9 vasuper domain^users 4.0K Sep 27 08:27 meta-webserver drwxr-xr-x 13 vasuper domain^users 4.0K Sep 27 08:27 meta-xfce -rw-r--r-- 1 vasuper domain^users 297 Sep 27 08:27 README We are going to be using only specific layers from the meta-openembedded layer as we don’t need all these yet. We’ll be using(the layers we choose here are detemined from reading the required dependencies from the layers used above this like meta-amd and meta-aws): meta-networking meta-python meta-filesystem meta-oe The meta-amd Layer: The meta-amd layer contains Board Support Packages (and a distro) for ONLY selected AMD x86 boards. The list of supported features on each of these supported boards is available here: https://git.yoctoproject.org/cgit/cgit.cgi/meta-amd/tree/FEATURES.md.\nYou can learn about the amd architectures that are supported in this path root/meta-amd-bsp/conf/machine in the meta-amd layer.\nThe meta-aws Layer: meta-aws adds useful aws bits for AWS IoT: awscli aws iot device client aws iot GGv2 aws sdk for python aws iot sdk for python aws cloudwatch publisher aws firecracker and more\u0026hellip;\u0026hellip; Other layers that you might need or can add: meta-rtlwifi for realtek wifi cards meta-virtualization for building Xen, KVM, Libvirt, and associated packages necessary for constructing OE-based virtualized solutions. meta-dpdk: Support layer for DPDK - a set of libraries and drivers for fast packet processing Setup: Make sure you have the build dependencies installed from the prerequisites section mentioned above.\nSelect the yocto branch (we’ll use the latest long term supported branch kirkstone)\n$ YOCTO_BRANCH=\u0026#34;kirkstone\u0026#34; Clone the git repos: $ git clone --single-branch --branch \u0026#34;${YOCTO_BRANCH}\u0026#34; \\ \u0026#34;git://git.yoctoproject.org/poky\u0026#34; \u0026#34;poky-amd-${YOCTO_BRANCH}\u0026#34; $ cd poky-amd-${YOCTO_BRANCH} $ git clone --single-branch --branch \u0026#34;${YOCTO_BRANCH}\u0026#34; \\ \u0026#34;git://git.openembedded.org/meta-openembedded\u0026#34; $ git clone --single-branch --branch \u0026#34;${YOCTO_BRANCH}\u0026#34; \\ \u0026#34;git://git.yoctoproject.org/meta-dpdk\u0026#34; $ git clone --single-branch --branch \u0026#34;${YOCTO_BRANCH}\u0026#34; \\ \u0026#34;https://github.com/aws4embeddedlinux/meta-aws\u0026#34; $ git clone --single-branch --branch \u0026#34;${YOCTO_BRANCH}\u0026#34; \\ \u0026#34;git://git.yoctoproject.org/meta-amd\u0026#34; Build: Select a target machine: You can find the supported platform types under the link below or the by traversing to the same path in the meta-amd-bsp git repo locally.\nhttps://git.yoctoproject.org/meta-amd/tree/meta-amd-bsp/conf/machine?h=kirkstone\nHere we\u0026rsquo;ll be using v1000.\nThis might support boards like too, but are not tested yet:\nAMD Ryzen Embedded V1000 (V-Series APU) AMD Ryzen Embedded R1000 (R-Series APU) AMD EPYC Embedded E3000 (E-Series CPU) You might ask, what are these machine codes? These are the code names AMD uses for their CPU architecture and each code distinguishes between stuff like Zen, Zen+, EPYC(milan/rome), Ryzen Embedded(v1000)etc, and for yocto we’d need to define these as part of the build.\nSetup the build environment and build for the selected machine: Source the oe-init-build-env script: $ source ./oe-init-build-env build-amd-v1000-awsiot-kirkstone Replace or edit accordingly the local.conf file from the one seen in this repo: https://github.com/Vasu77df/v1000-AMD-AWS-Yocto-Image\nadd the required layers to the build configuration:\n$ bitbake-layers add-layer ../meta-openembedded/meta-oe $ bitbake-layers add-layer ../meta-openembedded/meta-python $ bitbake-layers add-layer ../meta-openembedded/meta-networking $ bitbake-layers add-layer ../meta-dpdk $ bitbake-layers add-layer ../meta-aws $ bitbake-layers add-layer ../meta-amd/meta-amd-distro $ bitbake-layers add-layer ../meta-amd/meta-amd-bsp Start the build, by building one of the supported images core-image-base or core-image-sato . More on image types here: https://docs.yoctoproject.org/ref-manual/images.html#images I selected core-image-sato that uncludes x11 with the sato theme. bitbake core-image-sato -k the k || --continue flag will continue as much as possible after an error. While the target that failed and anything depending on it cannot be built, as much as possible will be built before stopping. Deploying an image to the target After building an image following the section above, we can deploy it to the target machine using a USB Flash Drive or a CD/DVD. The built images can be found in the \u0026lt;build-dir\u0026gt;/tmp/deploy/images/\u0026lt;machine-name\u0026gt; directory to which we will refer to as the \u0026ldquo;Image Deploy Directory\u0026rdquo; in this doc.\nNote:\nChange the \u0026lt;machine-name\u0026gt; and \u0026lt;image-name\u0026gt; placeholders in the following instructions according to the selected machine and the image built in following the section above. This directory contains .wic and .iso images for USB and CD/DVD respectively. Follow the instructions below to make a bootable USB Flash Drive or a CD/DVD by writing/burning the image to it:\nDeploy using a USB Flash Drive We can use bmaptool (from bmap-tools package) or dd to write the \u0026lt;image-name\u0026gt;-\u0026lt;machine-name\u0026gt;.wic image located in the Image Deploy Directory to a USB Flash Drive:\nUsing bmaptool (recommended) $ sudo bmaptool copy \u0026lt;image-name\u0026gt;-\u0026lt;machine-name\u0026gt;.wic /dev/\u0026lt;dev-node\u0026gt; where \u0026lt;dev-node\u0026gt; is to be replaced with the device node of the USB Flash Drive. (e.g. sda, sdb or sdc etc.) Booting the target Insert the bootable USB or CD/DVD (created in above steps) into the target machine and power ON the machine.\nThe grub boot menu should appear at this point where you will see options to boot or install this image:\nSelect the boot option to boot up the target machine.\nSelect the install option to install the image onto the target machine\u0026rsquo;s hard drive. Follow the instructions there to complete the installation process, and reboot the machine and boot from the hard drive you selected during the installation process.\nYou will be presented with a console (serial or graphical) or a graphical user interface depending on the image and the target machine.\nIf your t640 has a serial port, you can watch the boot messages. Eventually Linux will start a console on this device (i.e. the serial port). You can use a terminal emulator (such asminicom) to monitor the messages coming out of the serial port of your t640 on your host build machine.In my case I plugged in a serial-to-USB dongle into the t640 and my Linux host. On my Linux host the USB console device appears at /dev/ttyUSB2. Using /dev/ttyUSB2 as the port in minicom, and setting the communication parameters to 115200 8N1.\n","permalink":"https://vasuper.vasu3797.workers.dev/posts/amd-yocto-post/","summary":"\u003cp\u003eThis guide outlines the steps involved to build a linux distribution using Yocto for AMD Ryzen Embedded v1000 Series SoCs with AWS SDKs, services, and software from the \u003ca href=\"https://github.com/aws4embeddedlinux/meta-aws\"\u003emeta-aws\u003c/a\u003e layer.\u003c/p\u003e\n\u003ch2 id=\"prerequisites\"\u003ePrerequisites:\u003c/h2\u003e\n\u003cp\u003eTo follow the steps image build section, you will need:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003e\u003cstrong\u003eA Supported Linux Distribution\u003c/strong\u003e: You should have a reasonably current Linux-based host system. You will have the best results with a recent release of Fedora, openSUSE, Debian, Ubuntu, or CentOS as these releases are frequently tested against the Yocto Project® and officially supported. I’ve tried building this with both Ubuntu 20.04 and Pop_OS! 22.04.\u003c/p\u003e","title":"Building Yocto Based Linux Image for AMD Ryzen Embedded v1000 Series SoCs with AWS Agents installed."}]