Printing Hierarchical Directory Structures in Terminal

Tree is an open source command line tool for recursively outputting the structure of a directory. It is useful for generating a clear representation of complex directory structures.

Installation

Tree is compatible with most Unix based operating systems, including macOS. Since tree is not included with macOS, here are two methods for installing it:

Via Homebrew

If you already have Homebrew installed simply run:

brew install tree

From Source Code

1. Download the source code from GitHub:

git clone https://github.com/execjosh/tree.git

2. Move into the tree directory and edit the Makefile:

cd tree
nano Makefile

3. Comment out (prepend a #) the line under Linux defaults and uncomment (remove the #) the lines under the OS X section:

View fullsize

Note: To save changes in nano; press control + X, then Y and return.

4. To compile the binary simply run:

make

5. Move the newly generated binary into /usr/local/bin/:

sudo mv tree /usr/local/bin/

6. Lastly, move the manual page into /usr/share/man/man1/.

sudo mv doc/tree.1 /usr/share/man/man1/

Usage

Tree has many options and the manual page goes into each one in-depth, you can view the man page with:

man tree

Our favourite options:

• -C: Colour folder names to help distinguish files from folders
• -d: Only output directories, not files
• -H: Output as HTML with hyperlinks to files and folders
• -N: Do not escape spaces with forward slashes or replace non=printable characters
• -o: Send output to a file
• -Q: Put double quotes around filenames

Example

Using tree to list all files and folders in a user's Music directory:

View fullsize

Raspberry Pi Tips & Tricks

Introduction

For those unfamiliar, a Raspberry Pi is a low-cost, energy efficient, highly extensible, credit card sized computer. To assist new Raspberry Pi owners I have put together my notes on the topic and will continue to append to it as new discoveries are made.

Common Commands

Copying a Raspberry Pi Operating System Image (.img) to a microSD Card

Loading an initial operating system can seem like a daunting task, in fact many Mac users are unaware that copying an .img file to a microSD does not require any additional software. In this example I will be using Raspbian Stretch Lite, a good base image for most Raspberry Pi projects.

Connect the microSD card to a Mac using an SD card adapter or a microSD to USB reader.

To find the disk identifier of the microSD card, open Terminal and run the following command:

diskutil list

View fullsize

In my situation the microSD was mounted as /dev/disk2, I could tell this by comparing the size of the disk (31.1 GB).

Before we can copy the Raspbian data to the microSD card we need to unmount the disk first with the command:

sudo diskutil unmountDisk /dev/disk2

Time to copy the contents of the image file to the microSD card, take note of the r in front of disk2:

sudo dd if=~/Downloads/2017-11-29-raspbian-stretch-lite.img of=/dev/rdisk2 bs=1m

Setting Up a Headless (No Screen) Raspberry Pi

By default SSH (remote login) is disabled. To enable SSH create an empty file called "ssh" in the root of the SD card. On a Mac this can be achieved with the Terminal command:

touch /Volumes/boot/ssh

Connect to Wi-Fi

Raspbian includes the Raspberry Pi Software Configuration Tool, this includes a user friendly way to configure network settings. To open type:

sudo raspi-config

If you are unable to run the above command (e.g. no display or Ethernet available) and want the Raspberry Pi to connect to Wi-Fi, create a wpa_supplicant.conf file in the root of the SD card (/Volumes/boot/) and add the following:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=AU

network={
    ssid="Your Wi-Fi network name"
    psk="Your Wi-Fi password"
    key_mgmt=WPA-PSK
}

Changing the Default User Password

By default the username is pi and the password raspberry. Once logged into the Raspberry Pi, the easiest way to change the password is with the command:

sudo raspi-config

Updating Raspbian

To check for and install any available updates:

sudo apt-get update && sudo apt-get dist-upgrade && sudo reboot now

Favourite Use Cases

Pi-hole - Block Ads On Your Local Network

Pi-hole is one of the best uses of the Raspberry Pi hardware. Pi-hole acts as a Domain Name System (DNS) server on the local network, blocking requests to ad related networks. This results in webpages and apps displaying content without unwanted ads. The screenshots below show the difference Pi-hole makes to the website speedtest.net.

View fullsize

Before

View fullsize

After

Pi-hole also includes a nice dashboard, reporting usage and providing the ability to further blacklist/whitelist specific sites.

View fullsize

Digital Signage

Screenly & Yodeck have Raspberry Pi software allowing any TV to be used for digital signage. Updating what is displayed on screens is as easy as uploading new content via a web browser. Both currently offer a free tier for a single display.

Automate the Setup of Microsoft Exchange Accounts on OS X

I have been recently looking for the best way to automate the setup of Exchange accounts (specifically Office 365 hosted) on shared Macs. William Smith has created an impressive Exchange Setup AppleScript, perfect for Microsoft Outlook users.

I also wanted to automate the setup of Exchange accounts for Apple’s native OS X apps (Mail, Contacts, Calendars, Reminders and Notes). Normally this would be done with a Mobile Device Management (MDM) solution, pushing out user personalised configuration profiles. But for those situations where a MDM isn’t feasible (possibly due to budget, resources, policy, etc.) or simply overkill this post should help you out. 

To make life easier for those without a MDM I have put together a bash script to automate the setup of Exchange accounts on OS X.

How it works

The script locally generates and installs a user configuration profile (.mobileconfig file). To avoid the account being added as offline the user is also prompted for their Exchange account password.

Usage

I have tested this script on OS X El Capitan (10.11) with multiple Office 365 Exchange accounts.

1. Install Joseph Chilcote's Outset script.
2. Download the addexchangeaccount.sh script and customise the required DOMAIN and EXCHANGE_HOST values.
3. Then copy the customised script into /usr/local/outset/login-once/ and remember to make it executable.

That's it! The first time a user logs in they are prompted to enter their Exchange account password and then the script does the rest.

Deploy Finder's Sidebar List Favorites

Recently I came across Matt Schalk's Change_Sidebar_list.py script for interacting with Finder’s sidebar list favorites.

His script really caught my attention as I have come across situations where network home directories are missing the default set of Finder sidebar favorites.

To easily deploy a standard set of Finder sidebar favorites I have created another script (setsidebarfavorites.sh) that interfaces with Matt’s Change_Sidebar_list.py script. This script was also created with Joseph Chilcote's outset in mind. Placing setsidebarfavorites.sh in /usr/local/outset/login-once/ will initially configure a user’s favorites and then leave it up to them to further customise.

Usage

1. Install changesidebarlist-1.0.pkg

2. Install outset.pkg

3. Install setsidebarfavorites-1.0.pkg

customisation

If you want to further customise the standard list of favorites included in setsidebarfavorites-1.0.pkg download and edit setsidebarfavorites.sh.

View fullsize

Once customised simply copy setsidebarfavorites.sh into /usr/local/outset/login-once/ on your client machines and correct the script's file permissions with:

sudo chown root:wheel /usr/local/outset && chmod -R 755 /usr/local/outset && xattr -rc /usr/local/outset

Capturing Package Files with PkgKeeper

Deploying software via Munki is an excellent asset to sites managing fleets of Macs. Sometimes however, a package will not be listed directly on Apple's Support website and also may not be taking advantage of OS X Server's Caching Service. This is why I created the script PkgKeeper. The script works by monitoring filesystem access and if a pkg or dmg file is detected a hard link of the file is created on the user’s desktop.

At this point you may be asking yourself “what is a hard link?” Every unique file on a Unix (the foundation of OS X) filesystem has an inode (index node). One of the attributes of an inode is ‘link count.’ The link count is the number of hard links to a file.

Normally a file has a link count of just one, but when a new hard link is created that link count is incremented by one. Naturally, removing a file decrements the link count by one. It is not until the link count reaches zero that the inode is removed and the space is marked as available for use.

Under normal circumstances once an update package is installed and the package is removed the file's link count goes from one to zero. However, PkgKeeper creates another hard link of the file while it is still in use setting the file's link count to two. This stops the file from hitting a link count of zero and being completely removed.

Using The Script

Open Terminal and paste the following to download the script:

curl -O https://raw.githubusercontent.com/Error-freeIT/PkgKeeper/master/pkgkeeper.sh

Make the script executable:

chmod +x pkgkeeper.sh

Run the script:

sudo ./pkgkeeper.sh

Start downloading an update and watch as the script captures the package file.

View fullsize

Note for OS X 10.11 users: El Capitan's System Integrity Protection prevents this script from working. To temporally disable SIP boot into a recovery partition or 10.11 USB installer, open Terminal and type 'csrutil enable --without dtrace'.

Bonus Tips

In Terminal you can view a file's link count with the command:

stat -f '%l' FILE_NAME

The inode also contains the User ID, Group ID and file mode attributes of the file. Therefore all hard links will have the same user, group ownership and access permissions.

Once the update is installed the original process deletes its hard link to the file. This means it is no longer accessing the file and we are safe to edit the file's ownership. The easiest way to do this is by editing the 'Sharing & Permissions' section in the 'Get Info' window.

View fullsize

Installing & Automating AutoPkg with a Launch Daemon

UPDATE (1/12/15): Since AutoPkgr now supports recipe overrides I recommend using AutoPkgr over my headless script.

---

AutoPkg is a great tool for downloading and packaging software for distribution. It also integrates nicely with Munki. After testing AutoPkg I looked into methods to further automate the updating process. I came across Sean Kaiser’s blog detailing how he created a script and triggered it using a launch daemon. I took his script as an example and began to build my own, with a few extra tweaks.

My AutoPkg Wrapper triggers AutoPkg to run once daily at 8:30am, dynamically checking for updates to user recipe overrides and sending an email in the event of a new package.

Without further ado I will walk you through setting up AutoPkg and automating it with my AutoPkg Wrapper.

Note: If you haven’t already, it is important to install Munki and AutoPkg.

Configuring AutoPkg

First we need to add the main recipe repository to AutoPkg, we do this by opening Terminal and typing:

autopkg repo-add http://github.com/autopkg/recipes.git

Note: AutoPkg requires Git to be installed, if it is not installed you will be prompted to install it from Apple Software Update.

View fullsize

Next we need to set the location of our Munki repository:

defaults write com.github.autopkg MUNKI_REPO "/PATH/TO/MUNKI_REPO"

AutoPkg uses recipe overrides to override default or unspecified recipe attributes. My Munki repositories follow a consistent naming convention and therefore I create recipe overrides for every AutoPkg recipe used. Below are the criteria for packages added to my Munki repositories:

• Packages are placed directly into pkgs and not into subdirectories.
• Packages are named in lowercase (e.g. Firefox.dmg > firefox.dmg).
• Packages do not contain the developer’s name (e.g. googlechrome.dmg > chrome.dmg).
• Packages are initially added to the development catalog.
• The pkginfo display_name attribute is set to match the package name with proper case and spaces (e.g. flashplayer.dmg > Flash Player).
• The unattended_installs pkginfo attribute is disabled.
• The developer and category pkginfo attributes are set.

Optionally you can download and install my collection of AutoPkg recipe overrides from GitHub:

git clone https://github.com/Error-freeIT/AutoPkg-Recipe-Overrides.git ~/Library/AutoPkg/RecipeOverrides

With AutoPkg configured let’s test a recipe:

autopkg run -v AdobeFlashPlayer.munki

View fullsize

Automating AutoPkg

If that worked it’s time to automate AutoPkg, download and run the autopkgwrapper installer:

git clone https://github.com/Error-freeIT/AutoPkg-Wrapper.git /tmp/autopkgwrapper && cd /tmp/autopkgwrapper && sudo ./install.sh

The install script requires administrator privileges and therefore will prompt for a password, as it copies the script and launch daemon into place and opens the script in Nano for configuration.

Update the ACCOUNT_NAME value to match the account name (a.k.a. username) containing the recipe overrides and update the EMAIL_FROM and EMAIL_TO addresses.

Note: To save changes in the Nano text editor press control + X, type y and hit return.

View fullsize

That’s it! Once set up you will be emailed when new items are added to the Munki repository. Standard workflow would then involve testing the new software and if the new software is stable, simply add it to your production catalog.

Troubleshooting The AutoPkg Wrapper

This AutoPkg Wrapper was intended to only be run as a launch daemon (by root) to manually run the AutoPkg Wrapper type:

sudo "/Library/Scripts/AutoPkg Wrapper/autopkgwrapper.sh"

View fullsize

You can update your email settings by typing:

sudo nano "/Library/Scripts/AutoPkg Wrapper/autopkgwrapper.sh"

By default the installed launch daemon is set to run daily at 8:30am, if you want to further customise how often the script is run I recommend editing the launch daemon with Lingon X.

DeployStudio Rsync Backup & Restore User Data

The MacBook Air is a great laptop and thanks to its SSD (Solid-state Drive), it provides impressive read and write speeds. The only problem is that SSD storage is expensive and the approach of storing user data on a separate partition becomes unfeasible with the limited space. One site running a fleet of 64GB MacBook Airs found themselves manually backing up user data before reimaging, then manually restoring the user's data.

To help improve their workflow, I investigated methods of backing up user data during reimaging with DeployStudio. I came across the BackupRestore scripts by Rusty Myers and they looked great for backing up and restoring over a gigabit network (up to 125 MB/s). However, since all reimaging at this site occurs in the I.T. Office, I wanted to make use of the jaw-dropping speeds of USB 3.0 (up to 525 MB/s) and Thunderbolt (up to 1.25 GB/s). Of course, real world transfer rates are less due to disk read/write speeds, but it’s still a substantial improvement over gigabit Ethernet speeds.

My solution involves two scripts, a backup script ran just before DeployStudio reimages the internal disk and a restore script executed on first boot.

To further speed up the backup/restore process, unwanted data is skipped with rsync’s --include-from option, allowing pattern filtering of the items backed up and restored. A good example use is skipping the restore of ~/Library/Caches directories.

Setup

Setting up the required workflows is straightforward and takes approximately 10 minutes. To try it out simply follow the steps below.

Download backup.sh, restore.sh, backup_filter.txt and restore_filter.txt from GitHub here.

Copy backup.sh and restore.sh into your DeployStudio Scripts directory.

View fullsize

Copy backup_filter.txt and restore_filter.txt into your DeployStudio Files directory.

View fullsize

Open DeployStudio and create a workflow called 'Backup', add a ‘Generic’ script task and select backup.sh from the ‘Command’ drop-down list.

View fullsize

Create another workflow called ‘Restore’ and add a file copy task. Set the ‘Target volume’ to ‘Macintosh HD’, select ‘restore_filter.txt’ from the ‘File’ drop-down list and set the ‘Path’ to ‘/Library/Scripts’. Add a ‘Generic’ script task and select ‘restore.sh’ from the ‘Command’ drop-down list and check ‘Postponed execution (command will be launched on first boot).’

Create a third workflow called ‘Backup + Reimage + Restore’ and drag over three ‘Workflow’ tasks. As you can probably guess, the first ‘Workflow’ task is set to ‘Backup’. The second is your standard workflow for reimaging and the last is ‘Restore’.

Duplicate this workflow, rename it to ‘Reimage + Restore’ and remove the first backup ‘Workflow’ task. This workflow is used to restore a backup to a different Mac.

Lastly, uncheck the Publish checkboxes next to your reimage and ‘Restore’ workflows.

Time To Test

Before proceeding make sure you have a backup of any important data!

Format a USB/Thunderbolt/FireWire external disk, with the name 'Backups' and connect it to a Mac you wish to reimage.

View fullsize

NetBoot into DeployStudio and test out the new workflows.

Rapid Backup & Restore

Unfortunately, todays MacBook Airs only have one Thunderbolt port and during reimaging, that port is taken up by a Thunderbolt to gigabit Ethernet adapter. Although untested, I believe a Belkin Thunderbolt Express Dock would allow NetBooting into DeployStudio while attaching a Thunderbolt SSD.

Mac Migration

To migrate a user from one Mac to another, first run the the Backup workflow to take a backup of the source Mac. Connect the external disk to another Mac you will see a directory with the serial number of the source Mac. Simply rename the directory to the serial number of the destination Mac. Connect the backup disk to the destination Mac and then run a ‘Reimage + Restore’ workflow on the new Mac.

It is important to note that the restore script does not recreate user accounts, it just transfers user home directories. This is by design as the creation of the mobile account occurs when a user logs into the Mac for the first time.

Backup Archive

After successfully restoring a backup, it is moved into /Volumes/Backups/Restored/. This allows for recovery of any files that may have been skipped during restore, due to restore_filter.txt rules.

View fullsize