Sniffing packets on the Raspberry Pi 3 with FreeBSD, Netmap, and Suricata

I run Suricata on a Raspberry Pi 3 at home. Suricata is in af_packet mode, cluster type is cpu, and I have the ring size dialed in to consume most of the system’s memory. The source network isn’t of interest. Browsing traffic of a few people, mobile devices, streaming services, cat videos and garbage devices that clog up the tubes. The pi 3 surprisingly holds up well, for a device with a usb nic. Suricata drops < 3% of packets on average. It works. That’s fine and all, but I’ve been wondering how netmap on FreeBSD would function on restrictive and unsupported hardware. FreeBSD can run on the Raspberry Pi 3. Guess it’s time to find out.

Centralized syslog and storage

Logging data is good. Centralized logging is better. Making data available to a broad user base within your organization is critical. Several fantastic solutions exist (splunk, elk stack, graylog2), but implementing technology is not the end of the conversation. Policies and processes will dictate how logs are manipulated and archived. What type of data has value long term? Do you only need authentication logs for triage and incident response? Should kernel and custom application messages be stored longer to track timelines of overall degraded performance?

Coreboot on the lenovo x230

I’ve been a fan of the Thinkpad line since I first saw the little red dot mouse in the middle of the keyboard dubbed the “trackpoint” by IBM. Ten-ish years ago IBM’s personal computer line was sold off to Lenovo. The new owners have upheld the brand and continue to assemble decent hardware. They also has a long track record of not caring about privacy of end users and consistently backdoor the products they ship. This year alone they have been turned out for shipping back-doored versions of Windows, spyware that’s persistent in the bios, preloaded 3rd party applications that circumvents ssl, and now Lenovo is preloading fun tracking software. I have not (as far as I know) been impacted by the discovered and disclosed issues because I don’t run Windows on Lenovo hardware. This doesn’t mean FreeBSD or GNU/Linux users safe. It just means we have yet to be targeted (as far as we know) by Lenovo.

Creating a Scalable Graphite Cluster

There are a few decent posts on the internet about clustering graphite. Most examples I found were generally straight forward, but none of them addressed metric aggregation. My employer requires aggregation for various subsets of metric data. So be it. The write intensive ssd graphite servers in a mirrored configuration were bogged down from i/o saturation to the point of failing to render requests. We were also approaching the impending doom of storage constraints.

Automating Nagios with Puppet and Puppetdb

Monitoring automation should be a core component of any mission critical infrastructure. You can’t rely on servers and software. You can’t rely on people to correctly add servers and software to your monitoring system either. The only thing you can count on is that everything will fail at some point. Computers will always do a better job than humans, so outsource all you can to a machine.

For better or for worse Nagios has been affecting (controlling) my life for years. If you have spent any time looking at the puppetlabs type reference it’s hard to overlook that roughly a third of the types are for nagios objects. A clear indicator of how important this was to so many people to be migrated in as a core component of the product.

Separating data from code; using hiera in puppet

There are a few goals for writing puppet code that won’t crush your soul at a later date. Scalability across operating systems and environments, ease of maintenance, and writing “one manifest to rule them all” type code are pretty high on that list. Every time you add

if $::fqdn == ‘unique-flower.yourcubefarm.com’ {

to your puppet code, you are creating a condition where future administrators will want to murder you. Rightfully so. Over time your code base will grow from a handful of modules to a novel of state enforcement. The more conditional statements to support unique flowers, the harder it becomes to write or maintain manifests without similar conditional clauses, not to mention the increased time to spin up new hires. Seriously. Don’t mention it.

The truth about being a systems administrator

I often dream of a satisfying career that is stress free. One that consistently provides a feeling of achievement. A nine to five work schedule. Rare unplanned travel, long vacations, weekends off. A job where I can work on a project from start to finish, without interruption, change in focus, or it getting scrapped all together. Be employed at a place where I’m not questioned about things I don’t work with or called in for help on something I don’t know anything about. A job where title or description prevents me from being tasked with things that fall in line with another career.

Writing modular puppet modules

A problem I quickly ran into was controlling module behavior dependent on hostname in a scalable fashion. I burned through several failed attempts at implementing this in the module manifest using various “if” statements on facter variables. None of them were satisfactory. Luckily i floated back to puppet labs training documentation and found a modular approach was available. I then smacked myself in the face for overlooking it from the start. Twice.

Things not to do with puppet

I’ve recently had the pleasure to begin rolling out Puppet in various network enclaves. This can be both inviting and intimidating. On one hand, a sysadmin can see the light at the end of the tunnel, stepping closer to job automation. Puppet wears many hats, enabling easy change management for a large number of systems while functioning as an accurate time-line to resolve compliance audits at a moments notice. On the other hand, being tasked with writing manifests to define systems and their configurations across your employer’s environment could easily go sour. A simple change that should not have been made uniform or a tweak to correct a previous blunder can break application work flows. It’s a fine line between harnessing complexity, setting yourself adrift on an iceberg, or being thrown under the devops bus.

FreeBSD containers made easy

FreeBSD jails are far from new. This feature was committed to RELEASE in 2000 with 4.0-RELEASE. While the feature set has drastically grown, I rarely hear it mentioned in conversation discussing the pros and cons of VMware vs VirtualBox vs openVZ vs Linux-KVM.  Those familiar with system virtualization have most likely fiddled VMware,  VirtualBox or Linux-KVM. These hypervisors have the ability to emulate hardware environments and run os’es other than the host. Virtualization solutions like FreeBSD jails and OpenVZ operate differently. They take the monolithic approach, sharing the host kernel with the virtualized operating system container. This provides a low overhead solution, though neither are able to run a different operating system.

In assembly language we truss

Assembly language programming is almost a lost art. Piecing together opcode mnemonics from the x86 instruction set. Utilizing the interrupts, system calls, or api your operating system offers. Shaving off bytes and clock cycles for efficiency, elegance and bragging rights.

Debugging, loving and loathing.

The “pros” for assembly language have remained unchanged. Even if the code is poorly written it will produce a better optimized binary than any high level language compiler could spit out. No wasted resources. Few constraints from the operating system. Direct interaction with hardware. Blazingly fast software.

FreeBSD, Apache, MySQL, PHP = FAMP

A simple script to install Apache, MySQL and PHP on FreeBSD with userdir and server-status enabled. Warning, this may or may not be a sane configuration.

Due to a lawsuit from Bell Labs in the 90’s your peers are saying “LAMP” instead of “FAMP”. The latter acronym is far more hilarious. Please blurt it out at any chance you get.

While it looks like this, need to download it from here for it to work due to line breaks. I typed “please test before using in production”. This has been redacted. If you’re deploying FAMP stacks you already have a solution for implementation.

Simple arp poisoning resolution

Address resolution protocol poisoning is a problem which plagues most of us on switched ip v4 networks. Unfortunately this accounts for 95% of the worlds network infrastructure. Due to the limitations of ip v4, legacy installations, ease of use, lack of hardware solutions on low end gear, and lack of resources to maintain port security in high end gear, this 20 year old attack vector is all too common. Old problems lacking modern solutions. At least the Lawrence Berkeley National Laboratory provided us with a detection utility titled arpwatch, which has been available since 1992.

A brief analysis of a tor exit node

There is no doubt of the intentions of the tor project, though I’ve always wondered who most commonly uses it, and what kinds of traffic are they shuffling through it. In the event you’re unfamiliar with the tor project, it is a relay of encrypted proxies you may pass your network traffic through with the hopes of maintaining anonymity. You may find an in depth explanation of what tor is, how it works, and why it exists, here.

Who do we trust and why do we trust them?

Certificate authorities, because it’s currently our best option, despite it being a poor one.

Everyone who uses the internet relies on certificate authorities to authenticate somewhat important sites they visit. This is done with a model of public key cryptography, combined with a trusted-ish(ed) 3rd party entity acting as an escrow to delegate, maintain, and revoke cryptographic certificates. The commonly used browsers ship with a list of approved certificate authorities, which is not standardized, but chosen by the software vendor.

The "Unbreakable" Oracle, breaking everything.

A brand that lingers from the late 1970’s to current, It joins the ranks of IBM and Apple. The (Un)breakable Oracle now saddles an unfair share of the community. With their aggressive campaign and acquisition of Sun Microsystems, they control software which sends your tweets and makes your facebook google+ posts. They own the rights to their legacy propitiatory db products, mysql, innodb, java, openoffice, solaris, virtualbox, and now ksplice(1).

What is the big deal?

Oracle owns java. Java is a programming language that was designed so applications can be executed independent of platform. This is accomplished by the java runtime environment acting as a proxy, handling interactions with the operating system instead of the program itself. Lightweight virtualization. It’s not known for being fast, but it enables a developer to write a single program that functions on all the operating systems which the runtime environment is available. Java became extremely popular after its official release in 1995, being mostly used for web based client/server thing-a-ma-jigs and embedded nick-nacks. Chances are the cell phone in your pocket is a java based device, unless it’s a fruit phone, or made in 2003. This includes devices sporting the fast growing, open source, android operating system, which is developed by a small company called google.

Migrating FreeBSD to a new hard drive

I arrived at work to find a post-it note on my desk which read “Your disk was spinning a lot on Sunday”. The office was quiet, so a colleague could hear the disk spinning out of control. I leaned forward to hear the sound of a failing disk. High rpm spin up, slow down, and repeat. The usual behavior before the chug of death starts. Smartctl had reported roughly 30,000 seek errors in the past 20 minutes.

Using mutt with imap and imapfilter on FreeBSD

I generally prefer to use console applications over anything with a graphical user interface. I picked up this habit years ago from the desire to stuff everything I use in a screen session, accessible from a remote host. It still holds true that the console equivalent usually works just as well, runs faster, is easier to manage once configured without the need of reaching for a mouse and shuffling around windows, and generally sucks less.

Android Market. Holy f*ck its APT!

That’s right. The android OS market, now known as “advanced persistent threat”.

The buzz word from five years ago that basically defines “the internet”. This can also be defined as “an “intelligent” threat that is not going away, no matter how many safeguards we put in place”. One could also define this as “the way things have always been, and the way they will always be”. Unfortunately “everything is normal” does not get enough media attention to spur larger budgets to buy things you don’t need to pretend you’re mitigating threats you can’t actually reduce.

New weaponized exploit code for old systems

Why? Because it works.

Being something that resembles a pathetic excuse for an administrator, I’ve noticed a bit of downtime in the onslaught of root linux kernel exploits leaked over the past six months. The (re)release of cve-2010-3301 that Ben Hawkes (re)published resulted in an angry publication from Ac1dB1tCh3z of their full weaponized, and thoroughly badass implementation of cve-2010-3081. ABftw.c (code is full of lulz and a great read) was published to the full disclosure mailing list we have all grown to s/love/hate/, and within hours of the post companies everywhere were dealing with indiscreetly rooted servers, and mass website defacements. This exploit covered a broad range of linux kernels. The big winner is that most enterprises running gnu/linux servers deploy redhat / centos distributions. This vulnerability was mistakenly overlooked and backported into the kernel for these major distributions, allowing the attackers to maintain a working root exploit for years. Yes. Woops. I was not surprised a back-ported bug existed for a grip of gnu/linux distributions, though the publicly stated lifespan was impressive. The chaos development model strikes again.

Tracing malicious scripts on poorly configured gnu/linux servers

Warning: This post is worthless if you are not me.

I have recently been poking at a few old servers that still run apache handler 2.0. You got it. The good old days of all processes running as nobody from start to finish.

Accountability is for the birds.

These servers enjoy consistently being part of various botnets. Perl scripts are written to /tmp with random filenames (as the “nobody” user), executed by a call to the perl binary (not just as ./blah), the process forked, and the original file rm’ed. Insult to injury the procfs (which has been deprecated in FreeBSD in favor of procstat and sysctl entries are altered. This leaves little to trace as the usual task of using lsof against the pid will yield unproductive results on this configuration. You get the pid for a deleted file which has already been processed by perl, hence the entire perl exe is associated. As this particular altercation destroys the file descriptor, trying to cp the file via its /proc file descriptor results in zero bytes actually copied.

Stop poking the tiger

I can’t help but laugh as this has happened so recently after the gawker hacks. For those who don’t follow this kind of thing, the company publicly taunted a group of people who may or may not have ties to some people who may or may not also have ties to some people who may or may not allegedly be hackers. This stupid behavior resulted in the gawker being defaced. The social networking sites of the corporate entity and it’s employees were also popped, scrolling with sensitive personal data and assorted offensive one liners.

Password management and sinking to KeePassX

For the past several years I have been using pwman for password management. It’s an ncurses interface for an xml file encrypted with gpg. This program was an ideal solution. It provided an easy to use interface for a file encrypted with an industry standard utility, which I could use remotely, regardless if pwman was installed as long as I had my gpg key. Almost perfect.

Unfortunately pwman is unstable and tends to produce segmentation faults. This is unacceptable for regular use as most unix like systems have core dumps enabled, writing the entire password database in plain text to disk. Several conditions which result in a crash exist from adding, removing, and editing entries. There is one bug I’ve been able to easily replicate. Editing fields that end with a period (yes, a “.”) will result in a crash. A review of the decrypted xml file shows no corruption from the standard format, nor extra characters from the period. I have not flipped through the source to isolate and correct the bug.

ShmooCon triumphs and failures

Information security conferences are loved and hated. Defcon is a zoo. Blackhat and RSA are both a corporate oxymoron and expensive. There are now several others seemingly with ups and downs. Shmoocon passed and I had an interesting weekend spending too much time at the bar, being a bit inspired, and discovering a mild distaste for the failures of so called security professionals.

My only real problem with the conference was the timing of the ghost in the shellcode event. I was really hoping to spend more time on a hostile network getting my laptop owned by people who actually know what they are doing. Unfortunately you have to choose between being engaged with the game or seeing talks as they both finish at approximately at the same time each night.

HTTPS password sniffing on FreeBSD

Shmoocon is a few days away, it’s time to gear up with common tools that I rarely use. As a FreeBSD user sometimes I have to go through a little extra trouble to get some tools functioning correctly. It’s generally not on the top of the list as operating systems to use as an attack platform.

Fortunately I don’t have to manage wireless networks, nor do I have to watch exactly what data is traveling through http/s. So the top of the list is sniffing SSL traffic. Some people like dsniff, I like ettercap despite they have been owned and exposed, and the source possibly noodled with. Like any other software, run at your own risk, we are all most likely compromised anyway. Both software packages are available in the ports collection.

Resume

Director, Infrastructure - SpecterOps / January 2021 – Current

SpecterOps provided the opportunity of a greenfield architecture & implementation of infrastructure for BloodHound Enterprise.
I built the development and production environments needed to go to market. I spent my first year operating autonomously
as a Principal Infrastructure Engineer. My focus later expanded, bootstrapping multiple internal projects.

  • Manager of infrastructure team
  • AWS architecture & environment implementation
  • Maintaining development and production environments for a SaaS product and other offerings
  • Development in Go

Senior Manager, Infrastructure, Software Engineering – Gigamon / July 2018 - December 2020