"Unavailable console type spice" -- Another OpenStack Error

Tonight, I found myself trying to implement the spice proxy out of curiosity in havana. I found a nice post that talked about how to do this after banging my head on the wall for a while: http://joshrestivo.com/?p=32

Anyway, this lead to getting "console is currently unavailable. Please try again later. Reload" when I tried to load a console of an already running host. I dove into the stack trace I received in the nova-compute logs:

2014-01-07 22:41:51.402 5706 ERROR nova.openstack.common.rpc.amqp [req-4ad10aaf-60c0-4f88-8964-cb3f6dd06814 6c978326923a4fa997a6a83b3fdbd11e 47eedd8414b84466a731289a5d6dee35] Exception during message handling
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     **args)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py", line 439, in inner
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     return catch_client_exception(exceptions, func, *args, **kwargs)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py", line 420, in catch_client_exception
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     return func(*args, **kwargs)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 90, in wrapped
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     payload)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 73, in wrapped
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 271, in decorated_function
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     e, sys.exc_info())
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 258, in decorated_function
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3579, in get_spice_console
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     connect_info = self.driver.get_spice_console(instance)
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2199, in get_spice_console
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     ports = get_spice_ports_for_instance(instance['name'])
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2197, in get_spice_ports_for_instance
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp     raise exception.ConsoleTypeUnavailable(console_type='spice')
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp ConsoleTypeUnavailable: Unavailable console type spice.
2014-01-07 22:41:51.402 5706 TRACE nova.openstack.common.rpc.amqp 
2014-01-07 22:41:51.404 5706 ERROR nova.openstack.common.rpc.common [req-4ad10aaf-60c0-4f88-8964-cb3f6dd06814 6c978326923a4fa997a6a83b3fdbd11e 47eedd8414b84466a731289a5d6dee35] Returning exception Unavailable console type spice. to caller

and the final hint was a comment in the source:

# head -2197 /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py | tail -3

            # NOTE(rmk): We had Spice consoles enabled but the instance in

            # question is not actually listening for connections.

            raise exception.ConsoleTypeUnavailable(console_type='spice')

Turns out that running VMs have VNC or SPICE nailed into their configuration:

root@os-ph12:/# EDITOR=cat virsh edit instance-000000a1  | egrep 'graphics.*(spice|vnc)' 

   

root@os-ph12:/# EDITOR=cat virsh edit instance-000000a0  | egrep 'graphics.*(spice|vnc)' 

   

thus the VMs needed to be rebuilt on the host with spice.