Open Infrastructure Services LLC – OIS Blog

Blog: Privacy Policy

Tue, 23 May 2023 12:00:00 +0000

Last Updated: January, 2025

Privacy Policy of Open Infrastructure Services LLC

Overview

We at Open Infrastructure Services LLC respect your privacy and are committed to protecting your personal data. This privacy policy will inform you about how we look after your personal data when you visit our website (regardless of where you visit it from) and tell you about your privacy rights and how the law protects you.

Important Information and Who We Are

Open Infrastructure Services LLC provides custom software development and consulting services. This privacy policy aims to give you information on how Open Infrastructure Services LLC collects and processes your personal data through your use of this website, including any data you may provide through this website.

The Data We Collect About You

We may collect, use, store, and transfer different kinds of personal data about you, which we have grouped together as follows:

Identity Data includes first name, last name, username, or similar identifier.
Contact Data includes email address and telephone numbers.
Technical Data includes internet protocol (IP) address, your login data, browser type and version, time zone setting and location, browser plug-in types and versions, operating system and platform, and other technology on the devices you use to access this website.

Customer personal data for all of our products and services is transferred to the United States for login credential management and support both of which may be performed by US-based resources.

How Is Your Personal Data Collected?

We use different methods to collect data from and about you, including through:

Direct interactions. You may give us your Identity, Contact, and Financial Data by filling in forms or by corresponding with us by post, phone, email, or otherwise.
Automated technologies or interactions. As you interact with our website, we may automatically collect Technical Data about your equipment, browsing actions, and patterns.

How We Use Your Personal Data

We will only use your personal data when the law allows us to. Most commonly, we will use your personal data in the following circumstances:

Where we need to perform the contract we are about to enter into or have entered into with you.
Where it is necessary for our legitimate interests (or those of a third party) and your interests and fundamental rights do not override those interests.
Where we need to comply with a legal obligation.

Data Security

We have put in place appropriate security measures to prevent your personal data from being accidentally lost, used, or accessed in an unauthorized way, altered, or disclosed.

Your Rights

We recognize, under the EU-U.S. Data Privacy Framework Principles, the Swiss-U.S.Data Privacy Framework Principles, CCPA, the EU GDPR, the UK GDPR, the Brazilian General Data Protection Law (LGPD), and other applicable privacy laws, that you may have certain rights in regards to your Personal Information. We feel that your privacy and ability to preserve and exercise your rights is important. You are encouraged to review and understand these rights as they pertain to you and your Personal Information. In certain circumstances, these rights include, but are not limited to:

Right to be Informed: This means we have to tell you why we process your Personal Information, our retention periods, and who it will be shared with.
Right to Access: This means we have to provide you with a copy of your Personal Information we process upon your request.
Right to Rectification: This allows you to have inaccurate Personal Information rectified, or completed if it is incomplete.
Right to Erasure: This allows you to have your Personal Information erased.
Right to Restrict Processing: This means you can limit the way we use their data.
Right to Data Portability: This allows you to receive a copy of your Personal Information in a structured, commonly used and machine-readable format and gives you the right to transmit the data to another controller without hindrance.
Right to Object: This allows you to object to the processing of your Personal Information at any time.
Right to Non-Discrimination: The CPRA prohibits covered businesses from discriminating against consumers for exercising their CPRA rights. This means we cannot charge a different price, deny access to our products, or impose penalties for exercising your rights under the CPRA.
Right to Withdraw Consent: This means you can withdraw your consent at any time.

In support of these rights, you may exercise any of the above rights, with respect to your Personal Information. You may update, correct, or delete your Personal Information; if you wish to delete or suspend your account, please note that we may retain certain information as required by law or for legitimate business purposes. If you have become aware that an account has been created about you without your knowledge or consent, you may contact us to request deletion of that said account. You may contact us by emailing privacy@openinfrastructure.co

For your protection, we may only respond with the Personal Information associated with the particular email address that you use to send us your request, and we may need to verify your identity before implementing your request. We will respond to your request within 30 days.

Third-party Links

This website may include links to third-party websites, plug-ins, and applications. Clicking on those links or enabling those connections may allow third parties to collect or share data about you. We do not control these third-party websites and are not responsible for their privacy statements.

Third-Party Subprocessors

Available on Request

Contact Us

For any and all privacy-related matters, questions or comments, or to exercise a right under the EU’s GDPR, the UK’s GDPR or the CCPA, you may contact us in writing or by email. Our contact information is as follow:

Open Infrastructure Services, LLC

1675 SW MARLOW AVE STE 404

Portland, OR 97225

United States

Phone: +1 (669) 444-1014

Email: privacy@openinfrastructure.co

Website: https://openinfrastructure.co

In compliance with the EU-U.S. DPF and the UK Extension to the EU-U.S. DPF and the Swiss-U.S. DPF, Open Infrastructure Services LLC commits to resolve DPF Principles-related complaints about our collection and use of your personal information. EU and UK individuals and Swiss individuals with inquiries or complaints regarding our handling of personal data received in reliance on the EU-U.S. DPF and the UK Extension to the EU-U.S. DPF, and the Swiss-U.S. DPF should first contact Open Infrastructure Services LLC at: privacy@openinfrastructure.co.

If these processes do not result in a resolution, you may then contact your local data protection authority, the U.S. Department of Commerce, and/or the Federal Trade Commission for assistance.

Changes to the Privacy Policy

We may update this privacy policy from time to time. We encourage you to review this privacy policy frequently to be informed of how we are protecting your information.

Blog: FastAPI/Starlette OIDC Authentication

Tue, 28 Mar 2023 12:00:00 +0000

This post demonstrates how to validate a bearer token issued by an oidc provider. Assume the oauth flow is handled downstream. Starlette’s AuthenticationMiddleware is used to attach a SimpleUser and AuthCredentials instance to each request. Authlib validates the iss, exp, and aud claims. Starlette’s require decorator authorizes access token scopes.

# main.py
from fastapi import FastAPI, Request
from .middleware import middleware
from starlette.authentication import requires
from starlette.requests import Request


app = FastAPI(middleware=middleware)


@app.get("/")
async def homepage(request: Request):
 if request.user.is_authenticated:
 return {"username": request.user.username, "anonymous": False}
 return {"anonymous": True}


@app.get("/profile/")
@requires(scopes=["profile"])
async def profile(request: Request):
 return {"user": request.user, "auth": request.auth}

# middleware.py
import json
import logging
from functools import lru_cache
from threading import Timer
from urllib.request import urlopen

from authlib.jose.rfc7517.jwk import JsonWebKey, KeySet
from authlib.oauth2 import OAuth2Error, ResourceProtector
from authlib.oauth2.rfc6749 import MissingAuthorizationError
from authlib.oauth2.rfc7523 import JWTBearerTokenValidator
from authlib.oauth2.rfc7523.validator import JWTBearerToken
from fastapi import Request
from pydantic import BaseSettings, Field
from starlette.authentication import (
 AuthCredentials,
 AuthenticationBackend,
 AuthenticationError,
 SimpleUser,
)
from starlette.middleware import Middleware
from starlette.middleware.authentication import AuthenticationMiddleware
from starlette.requests import HTTPConnection, Request

logger = logging.getLogger(__name__)



class Settings(BaseSettings):
 secret_key: str = Field(title="Starlette SessionMiddleware Secret Key")
 oidc_issuer: str = Field(title="OIDC Issuer")
 oidc_audience: str = Field(title="OIDC Audience")

 class Config:
 env_file = ".env"

settings = Settings()


class RepeatTimer(Timer):
 def __init__(self, *args, **kwargs) -> None:
 super().__init__(*args, **kwargs)
 self.daemon = True

 def run(self):
 while not self.finished.wait(self.interval):
 self.function(*self.args, **self.kwargs)


class BearerTokenValidator(JWTBearerTokenValidator):
 def __init__(self, issuer: str, audience: str):
 self._issuer = issuer
 self._jwks_uri: str | None = None
 super().__init__(public_key=self.fetch_key(), issuer=issuer)
 self.claims_options = {
 "exp": {"essential": True},
 "aud": {"essential": True, "value": audience},
 "iss": {"essential": True, "value": issuer},
 }
 self._timer = RepeatTimer(3600, self.refresh)
 self._timer.start()

 def refresh(self):
 try:
 self.public_key = self.fetch_key()
 except Exception as exc:
 logger.warning(f"Could not update jwks public key: {exc}")

 def fetch_key(self) -> KeySet:
 """Fetch the jwks_uri document and return the KeySet."""
 response = urlopen(self.jwks_uri)
 logger.debug(f"OK GET {self.jwks_uri}")
 return JsonWebKey.import_key_set(json.loads(response.read()))

 @property
 def jwks_uri(self) -> str:
 """The jwks_uri field of the openid-configuration document."""
 if self._jwks_uri is None:
 config_url = urlopen(f"{self._issuer}/.well-known/openid-configuration")
 config = json.loads(config_url.read())
 self._jwks_uri = config["jwks_uri"]
 return self._jwks_uri


class BearerTokenAuthBackend(AuthenticationBackend):
 def __init__(self, issuer: str, audience: str) -> None:
 rp = ResourceProtector()
 validator = BearerTokenValidator(
 issuer=issuer,
 audience=audience,
 )
 rp.register_token_validator(validator)
 self.resource_protector = rp

 async def authenticate(self, conn: HTTPConnection):
 request = Request(conn.scope)
 try:
 token: JWTBearerToken = self.resource_protector.validate_request(
 scopes=["openid"],
 request=request,
 )
 except (MissingAuthorizationError, OAuth2Error) as error:
 raise AuthenticationError(error.description) from error
 scope: str = token.get_scope()
 scopes = scope.split()
 scopes.append("authenticated")
 return AuthCredentials(scopes=scopes), SimpleUser(username=token["email"])


middleware = [
 Middleware(
 AuthenticationMiddleware,
 backend=BearerTokenAuthBackend(
 issuer=settings.oidc_issuer,
 audience=settings.oidc_audience,
 ),
 )
]

Blog: New Puppet Service Offerings

Thu, 23 Feb 2023 12:00:00 +0000

TL;DR: Open Infrastructure Services (OIS) is proud to announce several new Puppet service offerings intended for customers of varying size, from hundreds to millions of managed nodes.

The commonality that unites the three of us at Open Infrastructure Services (OIS) is the fact that at some point in time, we all were members of Puppet Inc.’s Professional Services team. We were the engineers that were dispatched to solve the most complex customer issues, and together we have more than 25 years of experience in designing, customizing, and troubleshooting Puppet deployments. It’s that experience that allows us to solve real world problems in a predictable and timely manner, and we value our reputation of being able to deliver for our customers.

That’s why today we’re proud to announce several new Puppet service offerings available now to organizations of any size! While it’s not an exhaustive list, it does comprise some of the most popular features we’ve built in the past 6 years.

Safer Puppet Changes

Are Puppet changes causing incidents in your organization? Have you experienced service outages due to a Puppet code change which rolls out too quickly to control? Are you looking for a proven method to target a small group of nodes for a risky change, validate the results, then safely release it to the rest of your managed fleet?

This is a problem all large Puppet deployments need to solve, and we recently designed a solution for a large social media company that allowed them to reduce Puppet-related incidents from at least one per-month, to zero reported for more than an entire year.

The existing tools to solve this problem are ineffective at the scale of hundreds of thousands to millions of managed nodes. As a result, we developed a new, effective solution which:

Unlocks the ability to perform canary analysis on Puppet changes.
Reduces engineering time by adding scheduling policies to risky changes.
Leverages cloud native advancements in observability and traffic management.
Uses the same workflows as are used for application level container rollouts.

We’d love to share how our solution reduces the risk of production changes.

Advanced Reporting

Does your organization struggle with these questions about production changes?

When was a recent production change enforced on all managed nodes?
Was the production change successful or were there failures?
Was a recent Puppet change a contributing factor to a production incident?
What are the most common resource failures across teams?
Are nodes no longer being managed at all?

Many of these questions can be answered by the Puppet Enterprise Dashboard, but what if your organization doesn’t use Puppet Enterprise or has outgrown Puppet Enterprise? PuppetDB is open source and may help at smaller scales, but what if the number of nodes in your organization exceeds PuppetDB’s limits?

We recently built an advanced big data reporting solution using BigQuery and Data Studio to answer these questions. This solution worked well in production, scaling to hundreds of thousands of managed nodes and millions of Puppet run reports per day.

We’re available to tailor a reporting solution to the needs of your organization.

Module Development

The Puppet Forge has grown to contain thousands of Puppet modules managing as many applications and technologies, but it never seems to have the specific module that does what you need. And while the Puppet language (DSL) is legible and approachable, it’s VERY easy for a “simple fix” to trigger a cascade of unintended consequences.

Wouldn’t it be great if you could explain what you need and have someone write you a customized solution, conforming to best practices, complete with tests and USABLE documentation?

That’s exactly the kind of service we offer at OIS.

The Puppet experts at OIS have been writing Puppet modules since 2009, and have been following (and creating) best practices through to today. We literally wrote the book on the subject. Whether it’s writing a completely new module from scratch, or adding new functionality to existing code, we are here to help you work smarter, not harder.

Open Source Puppet Support

Free and open-source software (FOSS) is a bedrock principle at OIS - it puts the “Open” in “Open Infrastructure!” We also understand that customers using the FOSS version of Puppet have limited options when it comes to support plans. That’s why we offer first class support for ALL Puppet users, regardless of whether you’re using the Enterprise or FOSS editions. It doesn’t matter how old your code is or what your server stack looks like, we are here to help.

Whether you have a specific problem that needs a solution, or simply need the ability to contact the experts when problems arise, we can craft a support plan to fit your needs.

As we said above, this is not an exhaustive list of every Puppet service we’re able to offer. Do you have a problem that doesn’t fit neatly into one of the offerings above? Contact us and let’s start the conversation. Our most impactful engagements have started this way, and we’d be happy to discuss your goals.

Blog: Welcome Nate McCurdy

Mon, 03 Dec 2018 12:00:00 +0000

Open Infrastructure Services is growing! I’m excited to announce Nate McCurdy, has joined Open Infrastructure Services as a Cloud Infrastructure Consultant. Nate brings over a decade of experience helping companies with their digital transformation, configuration management, and breaking down walls between development and operations teams.

Nate is also a Google certified Cloud Architect and helping streamline large enterprise hybrid clouds to date.

Blog: Help Wanted

Wed, 18 Apr 2018 12:00:00 +0000

Open Infrastructure Services is hiring! We have a number of exciting projects on the horizon and are looking for talented infrastructure engineers. If you’re experienced with, or have an interest in cloud infrastructure automation and helping customers migrate their data center workloads to the public cloud, please send an email to careers@openinfrastructure.co. We’d love to talk to you about these exciting opportunities!

These projects are focused on large enterprise datacenter migrations to the Google Cloud.

Apply Now

Apply online now: Employment Application.

Blog: Update ESXi 6.5 to U1 over SSH

Wed, 22 Nov 2017 17:18:00 +0000

Updating ESXi 6.5 to 6.5 U1, I encountered the following error:

[root@esx:~] esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.5.0-20170702001-standard
[InstallationError]
[Errno 28] No space left on device
vibs = VMware_locker_tools-light_6.5.0-0.23.5969300
Please refer to the log file for more details.

The solution to this problem is to enable swap. I’m running this ESXi host on a single 32GB USB Thumb Drive, so I first had to create a VMFS5 datastore using the process at ESXi 6.5 Single USB Thumb Drive.

Once a datastore exists, enable Swap. Go to Host > System > Swap and activate swap on your datastore of choice. In my case there’s only one.

Once activated, this process to update ESXi over SSH worked flawlessly:

Enable outbound HTTP connections:

esxcli network firewall ruleset set -e true -r httpClient

Perform the update:

esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.5.0-20170702001-standard

Lock down HTTP connections after the update:

esxcli network firewall ruleset set -e false -r httpClient

Reboot the host:

reboot

For future reference, to see a list of available updates, use

esxcli software sources profile list -d \
https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml \
| awk '/6.5.0/ {print $1}'

[root@esx:~] esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | awk '/6.5.0/ {print $1}'
ESXi-6.5.0-20170701001s-no-tools
ESXi-6.5.0-20170404001-standard
ESXi-6.5.0-4564106-standard
ESXi-6.5.0-20170104001-standard
ESXi-6.5.0-20171004001-no-tools
ESXi-6.5.0-20170702001-no-tools
ESXi-6.5.0-20170404001-no-tools
ESXi-6.5.0-20170304101-no-tools
ESXi-6.5.0-20171004001-standard
ESXi-6.5.0-20170104001-no-tools
ESXi-6.5.0-4564106-no-tools
ESXi-6.5.0-20170304101-standard
ESXi-6.5.0-20170301001s-standard
ESXi-6.5.0-20170701001s-standard
ESXi-6.5.0-20170304001-standard
ESXi-6.5.0-20170702001-standard
ESXi-6.5.0-20170304001-no-tools
ESXi-6.5.0-20170301001s-no-tools

Blog: ESXi 6.5 Single USB Thumb Drive

Wed, 22 Nov 2017 15:06:00 +0000

I have a goal of booting an ESXi host from a single 32GB USB thumb drive. No other internal storage should be required for this firewall application. This is an ideal setup as there are no moving parts or cables to come unplugged. USB thumb drives are cheap and fast these days.

I was able to install ESXi 6.5 onto the USB thumb drive, but nothing shows up as an available data store for virtual machines. There’s a ton of free space on the USB stick. We can make use of this space with some partitioning magic.

I’m doing this all with a Mac OS X workstation. I’ll use a Ubuntu 16.04 VirtualBox instance to partition the USB stick. We’ll format the filesystem on the ESXi host itself.

First install, ESX onto the USB stick.

Shutdown the ESX host, remove the USB stick and insert into your Mac. Eject the disk from your mac so it can be passed through VirtualBox to Ubuntu 16.04:

sudo diskutil list
sudo diskutil eject disk1

To quickly get a Ubuntu desktop up and running, use vagrant:

mkdir ~/xenial
cd ~/xenial
vagrant init ubuntu/xenial64

Patch the vagrant file to get the GUI:

--- Vagrantfile.orig 2017-11-22 16:03:04.000000000 -0800
+++ Vagrantfile 2017-11-22 16:04:49.000000000 -0800
@@ -57,4 +57,8 @@
# vb.memory = "1024"
# end
+ config.vm.provider "virtualbox" do |vb|
+ vb.gui = true
+ vb.memory = "2048"
+ end
#
# View the documentation for the provider you are using for more

Bring up the vagrant instance:

vagrant up

Shutdown the instance and add USB to the virtual machine:

vagrant ssh -- sudo shutdown -h now

Go into VirtualBox Settings => Ports => Add a USB EHCI controller. Add a filter for the USB thumb drive. This is important, otherwise the USB thumb drive won’t show up in the Ubuntu VM. If the USB thub drive doesn’t show up in the GUI, make sure it’s been ejected from Mac OS X using diskutil eject prior to going into VirtualBox settings.

Install the Ubuntu Desktop:

vagrant ssh -- sudo apt-get install -y --no-install-recommends ubuntu-desktop

Install gparted

vagrant ssh -- sudo apt-get install -y gparted

Set a password for the user ubuntu:

vagrant ssh -- sudo passwd ubuntu

Reboot again to get the desktop up and running.

vagrant ssh -- sudo shutdown -r now

Use sudo gparted to create a new partition in the free space. Make sure to create it as unformatted, not the default of ext4.

Note the partition number, it should be partition 2, e.g. /dev/sdc2.

Use sudo gdisk /dev/sdc to change the partition type to fb00. The sequence here is:

t
2
fb00
w
Y

vagrant ssh -- sudo gdisk /dev/sdc
GPT fdisk (gdisk) version 1.0.1
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): t
Partition number (1-9): 2
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): fb00
Changed type of partition to 'VMWare VMFS'
Command (? for help): w
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!
Do you want to proceed? (Y/N): Y
OK; writing new GUID partition table (GPT) to /dev/sdc.

Finally, change partition 2 to partition 10 to avoid issues updating ESXi 6.5. The update process assumes partition 2 has not been created and will error out if present.

sudo sfdisk -d /dev/sdc > esxi.txt
cp -p esxi.txt esxi.txt.orig

Change esxi.txt as the following diff shows, moving partition 2 to 10.

--- esxi.txt.orig 2017-11-23 00:13:57.561990531 +0000
+++ esxi.txt 2017-11-23 00:15:35.566968530 +0000
@@ -7,5 +7,4 @@
/dev/sdc1 : start= 64, size= 8128, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=439DBC97-6DB2-4FD1-BDEA-A01FC9F26A49
-/dev/sdc2 : start= 8134656, size= 51927040, type=AA31E02A-400F-11DB-9590-000C2911D1B8, uuid=7661DD41-6B25-4ACA-9A7E-E68F07361B9E
/dev/sdc5 : start= 8224, size= 511968, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=CC0591B2-6658-4A25-91CF-1A9765D239A5
/dev/sdc6 : start= 520224, size= 511968, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=CDD8851F-3A51-47AD-80E1-F2D504197A8C
@@ -13,2 +12,3 @@
/dev/sdc8 : start= 1257504, size= 585696, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=3119D6C6-3EEC-4970-9289-6128686849EB
/dev/sdc9 : start= 1843200, size= 5242880, type=9D275380-40AD-11DB-BF97-000C2911D1B8, uuid=7A6D08A3-6E3F-488D-8F3B-36145382BA9F
+/dev/sdc10 : start= 8134656, size= 51927040, type=AA31E02A-400F-11DB-9590-000C2911D1B8, uuid=7661DD41-6B25-4ACA-9A7E-E68F07361B9E

Write the partition table back out to the USB drive:

sudo sfdisk --force /dev/sdc < esxi.txt

Check the partition table, make sure there is a partition 10:

sudo fdisk -l /dev/sdc

Insert the USB thumb drive back in the ESXi host and boot it back up. SSH in as root and check the partition table. There should be no partition 2 and you should see partition 10.

[root@esxi:~] partedUtil getptbl /dev/disks/mpx.vmhba32\:C0\:T0\:L0
gpt
3825 255 63 61457664
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B systemPartition 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
9 1843200 7086079 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
10 8134656 61456383 AA31E02A400F11DB9590000C2911D1B8 vmfs 0

Format the partition with vmkfstools -C vmfs5 -S USB.1:

[root@esxi:~] vmkfstools -C vmfs5 -S USB.1 /dev/disks/mpx.vmhba32\:C0\:T0\:L0:10
create fs deviceName:'/dev/disks/mpx.vmhba32:C0:T0:L0:10', fsShortName:'vmfs5', fsName:'USB.1'
deviceFullPath:/dev/disks/mpx.vmhba32:C0:T0:L0:10 deviceFile:mpx.vmhba32:C0:T0:L0:10
ATS on device /dev/disks/mpx.vmhba32:C0:T0:L0:10: not supported
.
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs5 file system on "mpx.vmhba32:C0:T0:L0:10" with blockSize 1048576 and volume label "USB.1".
Successfully created new volume: 5a1614ce-846cd3c8-9b10-0cc47aaaf624

The partition now shows up in the datastore browser after a refresh.

Configure swap, a persistent scratch location, and virtual machines on the same USB drive ESX is booting from, and enjoy!

Blog: Welcome Gary Larizza

Sat, 21 Oct 2017 12:06:00 +0000

Open Infrastructure Services is growing! I’m excited to announce Gary Larizza, has joined Open Infrastructure Services as a Principal Consultant. Gary and I first started working together on Puppet back in 2010 in Portland, Oregon. From that first day working together I was struck by Gary’s attitude and desire to dive deep in the guts of Puppet to figure out what’s really going on. Fast forward to 2017, and I’ve found myself visiting Shit Gary Says on a daily basis while working on Puppet types and providers. Gary’s ability to assimilate complex information, distill that information into the essential bits customers value, and communicate that essential information with humor and empathy has been invaluable to me personally. A great example of this depth, humor, and empathy is, Seriously, what is this provider doing?

Gary and I both started out working in education in Ohio, though separately from one another. We were bought together in Portland, OR by two australians from the other side of the world, Nigel Kersten and James Turnbull.

Gary and I share that special bond that only comes from successfully dealing with a classroom full of computers delivered a couple of weeks before school starts with the latest and greatest Mac OS X major release, chock full of changes and new surprises.

Please check out Gary’s post, Some Other Beginning’s End on this new beginning for both of us.

Contact us with your automation and cloud infrastructure goals. We’d both love to help you handle changes faster and more confidently through automated cloud infrastructure.

Gary is also a Puppet certified consultant and a member of the Puppet Services Delivery Partner Program. We’re here to help achieve your goals whether you’re expanding your Puppet investment, upgrading to the latest version and language features, or just getting started.

Blog: Puppet Enterprise Node Classification Backup & Restore with ncio

Sun, 21 Aug 2016 12:48:00 +0000

I’m happy to announce ncio, a small command line utility to backup and restore Puppet Enterprise Node Classification data.

A customer recently needed to automate the process of backing up and restoring Puppet Enterprise. Most of the work involved in accomplishing this goal is fairly straight-forward. The majority of the Puppet configuration is stored in version control in a Control Repository and Git is incredibly easy to backup and restore. Most customers I work with don’t mind losing data stored in PuppetDB because Puppet reports, resource information, and facts are automatically re-populated as nodes check in after the system is restored. The certificates used by Puppet are also fairly straight forward to backup as they live on the local filesystem.

The only service that was difficult to backup using normal filesystem tools is the Node Classification Service. The node classifier stores critical information and as a result it needs to be backed up and restored along side all of these other resources.

The Node classification v1 API is an excellent mechanism to retrieve and restore node classification data, but the task of using the API is largely left as an exercise for the reader. In order to help with this common problem, I wrote a small utility called ncio (node classification input / ouput). If you’d like to easily get a dump of all node classification data in pretty-printed JSON, transform a backup for restoration on a different PE Monolithic Master, or restore a backup then this tool is for you.

The tool is distributed and updated on rubygems in an effort to make it easiy to install and upgrade in the future.

Here’s how to get started:

Installation

Installation is straight-forward thanks to Puppet Enterprise shipping with the gem command:

$ sudo /opt/puppetlabs/puppet/bin/gem install ncio
Successfully installed ncio-1.1.0
Parsing documentation for ncio-1.1.0
Installing ri documentation for ncio-1.1.0
Done installing documentation for ncio after 0 seconds
1 gem installed

Usage

The command runs best from a PE Monolithic Master. The tool automatically uses SSL certificates which are already present to make setup as easy as possible.

sudo -H -u pe-puppet /opt/puppetlabs/puppet/bin/ncio backup > /var/tmp/backup.json
I, [2016-06-28T19:25:55.507684 #2992] INFO -- : Backup completed successfully!

Retrying Connections

When automating a backup from cron, it’s recommended to use the --retry-connections global option to make the backup as robust as possible. This option allows ncio to retry in certain situations, e.g. when the puppetserver service is restarting.

Restore Pipelines

The command is designed to send a backup to standard output and restore from standard input. This allows backup and restore pipelines. In this example a backup is taken on master1, transformed so that it can be restored on master2, then restored on master2:

export PATH="/opt/pupeptlabs/puppet/bin:$PATH"
ncio --uri https://master1.puppet.vm:4433/classification-api/v1 backup \
 | ncio transform --hostname master1.puppet.vm:master2.puppet.vm \
 | ncio --uri https://master2.puppet.vm:4433/classification-api/v1 restore

Command overview

Global options:

$ ncio --help
usage: ncio [GLOBAL OPTIONS] SUBCOMMAND [ARGS]
Sub Commands:
backup Backup Node Classification resources
restore Restore Node Classification resources
transform Transform a backup, replacing hostnames
Quick Start: On the host of the Node Classifier service, as root or pe-puppet
(to read certs and keys)
/opt/puppetlabs/puppet/bin/ncio backup > groups.$(date +%s).json
/opt/puppetlabs/puppet/bin/ncio restore < groups.1467151827.json
Transformation:
ncio --uri https://master1.puppet.vm:4433/classification-api/v1 backup \
| ncio transform --hostname master1.puppet.vm:master2.puppet.vm \
| ncio --uri https://master2.puppet.vm:4433/classification-api/v1 restore
Global options: (Note, command line arguments supersede ENV vars in {}'s)
-u, --uri=<s> Node Classifier service uri {NCIO_URI}
(default: https://localhost:4433/classifier-api/v1)
-c, --cert=<s> White listed client SSL cert {NCIO_CERT}
See: https://goo.gl/zCjncC (default:
/etc/puppetlabs/puppet/ssl/certs/pe-internal-orchestrator.pem)
-k, --key=<s> Client RSA key, must match certificate {NCIO_KEY} (default:
/etc/puppetlabs/puppet/ssl/private_keys/pe-internal-orchestrator.pem)
-a, --cacert=<s> CA Cert to authenticate the service uri {NCIO_CACERT}
(default: /etc/puppetlabs/puppet/ssl/certs/ca.pem)
-l, --logto=<s> Log file to write to or keywords STDOUT,
STDERR {NCIO_LOGTO} (default: STDERR)
-s, --syslog, --no-syslog Log to syslog (default: true)
-v, --verbose Set log level to INFO
-d, --debug Set log level to DEBUG
-r, --retry-connections Retry API connections, e.g. waiting for the
service to come online. {NCIO_RETRY_CONNECTIONS}
-o, --connect-timeout=<i> Retry <i> seconds if --retry-connections=true
{NCIO_CONNECT_TIMEOUT} (default: 120)
-e, --version Print version and exit
-h, --help Show this message

Backup options

$ ncio backup --help
Node Classification backup options:
-g, --groups, --no-groups Operate against NC groups. See: https://goo.gl/QD6ZdW (default: true)
-f, --file=<s> File to operate against {NCIO_FILE} or STDOUT, STDERR (default: STDOUT)
-h, --help Show this message

Transform options

$ ncio transform --help
Node Classification transformations
Note: Currently only Monolithic (All-in-one) deployments are supported.
Transformation matches against class names assigned to groups. Transformation
of hostnames happen against rules assigned to groups and class parameters for
matching classes.
Options:
-c, --class-matcher=<s> Regexp matching classes assigned to groups.
Passed to Regexp.new() (default: ^puppet_enterprise)
-i, --input=<s> Input file path or keywords STDIN, STDOUT, STDERR (default: STDIN)
-o, --output=<s> Output file path or keywords STDIN, STDOUT, STDERR (default: STDOUT)
-h, --hostname=<s+> Replace the fully qualified domain name on the left with the
right, separated with a :
e.g --hostname master1.acme.com:master2.acme.com
-e, --help Show this message

Restore options

$ ncio restore --help
Node Classification restore options:
-g, --groups, --no-groups Operate against NC groups.
See: https://goo.gl/QD6ZdW (default: true)
-f, --file=<s> File to operate against {NCIO_FILE} or STDOUT,
STDERR (default: STDIN)
-h, --help Show this message

Hopefully you find ncio useful. If so, please let me know! If you run into any issues or would like to see additional features, please open up an issue on the project page.

Blog: Logo

Sat, 27 Feb 2016 17:53:00 +0000

Finally got a logo. It turned out to be pretty easy once I finally took the plunge and signed up for fiverr. Fairly inexpensive, and a quick 5 day turnaround on the design. I was able to take the original image files and work with them in The GNU Image Manipulation Program.

Blog: Lift Off!

Tue, 06 Oct 2015 13:39:00 +0000

3.. 2.. 1.. Lift Off!

I’m pleased to announce Open Infrastructure Services, LLC is open for business! Here’s some information about myself and the services I have available. Please email me with any questions about how I can help.

Open Infrastructure Services is founded by me, Jeff McCune, a veteran systems administrator and software developer with over 15 years of experience. Before starting my consulting business I was at Puppet Labs as a software developer working on Puppet, Facter, Puppet Server, and other internal tools like crossfader. Prior to that, I was a professional services engineer at Puppet Labs, traveling the world working with customers and the open source community on matters of system architecture, availability, security, and continuous delivery workflows.

While traveling for Puppet Labs, I also co-authored Pro Puppet, the best-selling book about how to quickly get started and then scale Puppet toward full automation.

Before Puppet Labs I worked at Netsmart Technologies where I was responsible for our hosted SaaS datacenters. Puppet and DevOps practices played a key part of my success at Netsmart. We were able to implement SAS-70 II compliance in a matter of months using a well defined change management process, and detailed automated reports integrated with Puppet and Splunk.

Prior to Netsmart Technologies, I worked at the Mathematics Department at Ohio State University. There, I learned about automation, cfengine, started using Puppet, and fully automated our Mac OS X workstations and labs. In order to achieve full automation for PowerPC and Intel based systems, I reverse engineered Apple’s boot service discovery protocol and patched ISC DHCP to behave as an automated network boot server for Mac OS X. I presented on Puppet at Apple’s Worldwide Developer Conference.

Please contact me if you have a need for professional, intelligent and creative solutions to the infrastructure problems that are holding you back from delivering your applications.

Blog: Announcing Open Infrastructure Services, LLC

Sun, 20 Sep 2015 18:33:10 +0000

Hello, World!

I’m a little sad to say I’ve left Puppet Labs after over 5 years working for the company and working on Puppet. It was a difficult decision, but I’m pleased to announce that I’ve decided to start my own independent consulting business, Open Infrastructure Services, LLC. I’ll be helping clients with their Puppet related problem while also focusing on bringing creative solutions to bear on complex infrastructures. Please email me for more information, I’d love to hear how I could help you and your business.

company { "Open Infrastructure Services, LLC":
 ensure => running,
 founder => "Jeff McCune",
 website => "http://openinfrastructure.co",
 email => "jeff@openinfrastructure.co",
}