devstack in a lxd container
When doing Openstack development, it's hard to avoid using devstack to build and maintain a development environment. While devstack is a neat tool, it can't possibly handle every single case and environment and OS and network configuration. At least, it's very difficult, and requires no end of (sometimes archaic) configuration settings. This is all to say that I appreciate those that work on devstack, but I often find myself futzing more with devstack than actually writing code, and I find this less than optimal.
As I've been working an nova-lxd I
continue to run into weird networking issues, mostly related to neutron bridges and
openvswitch. I got into a weird case where devstack's
stack.sh script couldn't divine
what interfaces need be connected to openvswitch bridge, etc. This meant that I could fire
up nova instances, but I could never access them, because the routing never worked.
While most other developers turn to virtual machines, it seemed a little counterproductive that I'm working on the LXD hypervisor for Openstack Nova. Naturally, I turned to trying devstack in a lxd container.
(I'm using Ubuntu 16.04; varying mileage and all that)
$ lxc launch ubuntu:16.04 devstack -c security.privileged=true -c security.nesting=true $ lxc exec devstack -- apt-get install -y avahi-daemon
In this case, you need
security.nesting to fire up lxd instances (which is required for
me, as I'm developing nova-lxd). I'm not sure whether I need
but it certainly made things a little easier for me. I install
avahi-daemon so I can do
ssh devstack.local without remembering ip addresses.
At this point, I created an account in the container that is identical to my current
username (which, as you may infer, is
rockstar). You could create a
stack user if
you're one of those people that creates a specific devstack user.
$ user=`whoami` $ lxc exec devstack -- adduser --disabled-password --gecos "" $user
...and to make things easier in my development environment, I added my home directory into the container.
$ lxc config device add devstack homedir disk source=/home/$user path=/home/$user
And now, the devstack stuff.
$ git clone https://git.openstack.org/openstack-dev/devstack
At this point, you've got a container with a real
eth0, instead of some sort of
ens0lmnopwftbbq random device name. Here's my
[[local|localrc]] HOST_IP=10.193.59.173 # set this to your IP FLAT_INTERFACE=eth0 ADMIN_PASSWORD=password DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=$ADMIN_PASSWORD ENABLED_SERVICES=rabbit,mysql,key ENABLED_SERVICES+=,g-api,g-reg ENABLED_SERVICES+=,n-cpu,n-api,n-crt,n-obj,n-cond,n-sch,n-novnc,n-cauth ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-meta,q-l3 disable_service n-net enable_plugin nova-lxd https://git.openstack.org/openstack/nova-lxd
We explicitly disable nova-network (it doesn't really work with nova-lxd). I don't use horizon at all, and cinder requires some extra settings container configuration, as it requires lvm (I don't know the specifics here; I may explore it later).
Once you're all configured, run
./stack.sh and watch the world burn (for a bit). Once
complete, if you're like me and are running nova-lxd, you can fire up a nova instance with
$ OS_USERNAME=admin $ OS_PASSWORD=password $ OS_PROJECT_NAME=demo $ OS_AUTH_URL=http://10.193.59.173/identity $ $ wget http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-root.tar.gz $ openstack image create --disk-format raw --container-format bare --file xenial-server-cloudimg-amd64-root.tar.gz xenial $ openstack server create --image xenial --flavor m1.tiny test-container