devstack in a lxd container
When doing Openstack development, it's hard to avoid using devstack to build and maintain a development environment. While devstack is a neat tool, it can't possibly handle every single case and environment and OS and network configuration. At least, it's very difficult, and requires no end of (sometimes archaic) configuration settings. This is all to say that I appreciate those that work on devstack, but I often find myself futzing more with devstack than actually writing code, and I find this less than optimal.
As I've been working an nova-lxd I
continue to run into weird networking issues, mostly related to neutron bridges and
openvswitch. I got into a weird case where devstack's stack.sh
script couldn't divine
what interfaces need be connected to openvswitch bridge, etc. This meant that I could fire
up nova instances, but I could never access them, because the routing never worked.
While most other developers turn to virtual machines, it seemed a little counterproductive that I'm working on the LXD hypervisor for Openstack Nova. Naturally, I turned to trying devstack in a lxd container.
(I'm using Ubuntu 16.04; varying mileage and all that)
$ lxc launch ubuntu:16.04 devstack -c security.privileged=true -c security.nesting=true
$ lxc exec devstack -- apt-get install -y avahi-daemon
In this case, you need security.nesting
to fire up lxd instances (which is required for
me, as I'm developing nova-lxd). I'm not sure whether I need security.privileged
here,
but it certainly made things a little easier for me. I install avahi-daemon
so I can do
ssh devstack.local
without remembering ip addresses.
At this point, I created an account in the container that is identical to my current
username (which, as you may infer, is rockstar
). You could create a stack
user if
you're one of those people that creates a specific devstack user.
$ user=`whoami`
$ lxc exec devstack -- adduser --disabled-password --gecos "" $user
...and to make things easier in my development environment, I added my home directory into the container.
$ lxc config device add devstack homedir disk source=/home/$user path=/home/$user
And now, the devstack stuff.
$ git clone https://git.openstack.org/openstack-dev/devstack
At this point, you've got a container with a real eth0
, instead of some sort of
ens0lmnopwftbbq
random device name. Here's my local.conf
:
[[local|localrc]]
HOST_IP=10.193.59.173 # set this to your IP
FLAT_INTERFACE=eth0
ADMIN_PASSWORD=password
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=$ADMIN_PASSWORD
ENABLED_SERVICES=rabbit,mysql,key
ENABLED_SERVICES+=,g-api,g-reg
ENABLED_SERVICES+=,n-cpu,n-api,n-crt,n-obj,n-cond,n-sch,n-novnc,n-cauth
ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-meta,q-l3
disable_service n-net
enable_plugin nova-lxd https://git.openstack.org/openstack/nova-lxd
We explicitly disable nova-network (it doesn't really work with nova-lxd). I don't use horizon at all, and cinder requires some extra settings container configuration, as it requires lvm (I don't know the specifics here; I may explore it later).
Once you're all configured, run ./stack.sh
and watch the world burn (for a bit). Once
complete, if you're like me and are running nova-lxd, you can fire up a nova instance with
the following.
$ OS_USERNAME=admin
$ OS_PASSWORD=password
$ OS_PROJECT_NAME=demo
$ OS_AUTH_URL=http://10.193.59.173/identity
$
$ wget http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-root.tar.gz
$ openstack image create --disk-format raw --container-format bare --file xenial-server-cloudimg-amd64-root.tar.gz xenial
$ openstack server create --image xenial --flavor m1.tiny test-container