Running SingleStore on Apple Silicon

This guide will take you through installing SingleStore on your Apple Silicon machine for development purposes. SingleStore are working on building official support for development-only workloads, meaning that this solution involving UTM/QEMU is not officially tested or supported by SingleStore.

SingleStore is a modern-day database solution built for rapid aggregation within your data-intensive applications. It is much faster at fetching and aggregating data than traditional MySQL-based solutions. It has personally been a crucial part of my gaming-focused analytics platform, so let me give some background to who I am and why I wrote this guide.

I'm Charlie Joseph and I run a Minecraft server analytics platform called Analyse. I built Analyse as I have ran and managed various gaming communities, and felt frustrated with how clunky and memory heavy other solutions were so I built my own. Analytics are a fundamental part of understanding your gaming servers growth, and building Analyse meant I knew I'd be dealing with high volumes of data from launch. SingleStore has been a fantastic solution to quickly aggregate data, which lead me to want to host it locally so I can experiment further with it.

#Installation

For this guide we will be creating a Debian-based virtual machine which will host our SingleStore database, for this you'll need to download the Debian ISO alongside UTM to run our virtual machine with Apple Silicon.

Once you've downloaded these files, drag the UTM application into your Macs applications folder and then open the application.

#Setting up

With the UTM application open, press the + symbol found at the top of the application to create a new machine. Then..

1. Select the "Emulate" option followed by the "Linux" option.
2. Then select "Browse" and choose the Debian ISO that we downloaded earlier and click "Continue".
3. Now under "Architecture" ensure that x86_64 is selected and Standard PC (Q35 + ICH9, 2009) (alias of pc-q35-7.0) (q35) is selected under "System".
4. Under "Memory" it is recommended by SingleStore to have at least 4GB of ram selected, so I'd at least go with 4096. You can optionally specify how many CPU cores you wish to allocate, however I left this option as default. Then press "Continue".
5. For our machines disk space, it is recommended to assign around 100GB however you can go slightly lower. Then press "Continue".
6. Once you're on the "Shared Directory" page press continue again.
7. Finally you'll be taken to the "Summary" page where you can specify a name for the virtual machine, I went with SingleStore for this guide. Then tick the Open VM Options checkbox and press "Save".

#Enabling networking

Now we've got our virtual machine created and the settings page open, we will configure networking so our machine will have internet access. So..

1. Select the "Network" tab from the side-bar.
2. Under the "Network Mode" dropdown select the Emulated VLAN option.
3. Then press "Save".

#Starting our machine

With our machine created and our networking configured, it is finally time to begin booting it so that we can install our virtual machine. So..

1. Press the start button next to your virtual machine.
2. Once booted select the "Graphical Install" option.

#Installing Debian

Once the menu has loaded you'll be greeted with a Debian 11 banner at the top. Once you see this..

1. Select your language using the UP and DOWN keys and then press ENTER to choose that option. I chose "English".
2. Then select your location and press ENTER, I'm British so I went with "United Kingdom".
3. Now select your keyboard layout and press ENTER. I chose "British English".
4. You'll need to wait roughly 5 minutes for the installation media to load until you see "Configure the network".
5. Then choose the hostname that you wish to appear on your local network and press ENTER. I chose "harleyquinn".
6. Once you see the "Domain Name" option, just press ENTER.
7. Next you'll be asked to create a password for the root user, as this is for development purposes and then press ENTER. I chose "password".
8. Now you'll be asked to create a new user alongside root:
   1. For the "Full Name" choose what you'd like to be called.
   2. For the "Username" choose which username you'd like.
   3. For the "Password" choose which password to use, I went with the same as above.
   4. Finally press the ENTER key.
9. Now wait until you see the "Partitions Disks" page. choose the Guided - use entire disk option and press ENTER.
10. Select the ATA QEMU HARDDISK option and press ENTER.
11. Select the All files in one partition option and press enter.
12. Select the Finish partitioning and write changes to disk option and press ENTER.
13. Select the Yes option under "Write the changes to disk?" and press ENTER.
14. Now wait for the installation to complete which takes around 10 minutes.

#Configuring the package manager

Once the installation is complete you'll see the "Configure the package manager" screen, once you're here..

1. Select No for the "Scan extra installation media" and press ENTER.
2. Select your mirror location using the UP and DOWN keys and then press ENTER to choose that option. I chose "United Kingdom".
3. Under "Debian archive mirror" ensure deb.debian.org is selected and press ENTER.
4. For "HTTP proxy information" leave this blank and press ENTER.
5. After a few minutes you'll see the "Configuring popularity-contest" page, select No and press ENTER.
6. Then untick both Debian desktop environment, GNOME and standard system utilities, and tick SSH server and press ENTER.
7. Now wait for the "Installation complete" message.

#Ejecting the ISO

With installation complete we can now eject the media like so..

1. Select the CD icon in the top right and hover over CD/DVD.
2. Then press the Eject button.
3. Finally select continue from the installer.

#Configuring local portforwarding

To make set-up easier alongside allowing us to access our SingleStore instance locally from our Mac, we will configure the ports from our machine. To do this..

1. Click the power icon in the top left to power the machine off.
2. Right click the virtual machine and select Edit.
3. Go to the "Networking" tab.
4. Beside "Port Forward" select the new tab.
5. Select 22 for the "Guest Port" and 2222 for the "Host Port" so we can SSH in locally and click save.
6. Now do the same for port 3306 (database) and 8080 for SingleStores viewer. If you wish to use a different port for any of these, ensure that you only change the "Host Port" value.
7. Now click "Save" at the bottom.

#Configuring SingleStore

With our machine installed and configured we can finally begin installing SingleStore. Press the start button beside the machine option. Then..

1. Click the start button to launch the machine.
2. Once booted, enter root for the username.
3. Enter the password you created earlier.

#Configuring SSH for our Mac terminal

Now we're logged in, we need to configure SSH so we can paste from our Mac to our machine. To do this, you'll need to type the following commands into the terminal:

1. Run nano '/etc/ssh/sshd_config.
2. Add PermitRootLogin yes to the file and then press CTRL + X and then press Y to save the file.
3. Finally, type systemctl restart ssh to restart our SSH server.

#Installing SingleStore

Now that we've enabled SSH, open your Macs terminal (you can use CMD + SPACE and then typing "Terminal" to do this). Then..

1. Run ssh -p 2222 root@localhost (or change 2222 for the other SSH port we configured earlier).
2. Enter your password you configured earlier and press ENTER.

Now that we're logged in, let's begin:

1. Run apt -y install apt-transport-https gnupg to install our basic packages.
2. Then wget -O - 'https://release.memsql.com/release-aug2018.gpg' 2>/dev/null | apt-key add - && apt-key list to add the SingleStore repo.
3. Then echo "deb [arch=amd64] https://release.memsql.com/production/debian memsql main" | tee /etc/apt/sources.list.d/memsql.list to add the package.
4. Finally apt update && apt -y install singlestoredb-toolbox singlestore-client singlestoredb-studio to install SingleStore.

#Creating our cluster

With SingleStore added, we can finally create our new cluster. To do this, head to your SingleStore Portal and sign in. Then..

1. Click "Org:" on the left hand side.
2. Select the "On-Prem Licenses" option.
3. In your terminal, run LICENSE_KEY= followed by the license key found under "Free License" on the SingleStore portal.
4. Now lets configure a database password by using LOCAL_PASSWORD= followed by your chosen password.
5. Now run sdb-deploy cluster-in-a-box --license $LICENSE_KEY --version 7.8 --password $LOCAL_PASSWORD --bind-address 0.0.0.0 to install our cluster.
6. On the prompts, type y and y again for both options.

#Configuring our cluster

Now our cluster has been created, we need to ensure it starts on boot alongside starting our cluster now. Which can be done like so..

1. Run systemctl enable singlestoredb-studio.service
2. Run systemctl start singlestoredb-studio.service

#Logging into SingleStore Studio

With our cluster configured and started up, head to your browser and go to http://localhost:8080 (or another port if configured earlier). Once here, we'll need to..

1. Select "Add Existing Cluster".
2. Type 0.0.0.0 for the "IP Address".
3. Type 3306 for the "Port".
4. For the login details..
   1. Use root for the "Username".
   2. Use the password you specified for LOCAL_PASSWORD above.
5. Select Development for the "Type".
6. For the final details..
   1. Choose a custom name for it.
   2. Optionally specify a memorable description.
7. Finally, click "Create".

Now you can login to your cluster for the online portal, and connect to your database locally using port 3306.

This was a long blog post, if it helped you out, be sure to tweet at me @heychazza.