* ceph by vote - but will accept multiple options/ HA next steps: We have working ground-> overcloud now. But it's not HA! Lets get that done. In particular we need to identify all the things to HA and talk through single node + 2 node + 3+ node setups of them. OpenStack HA guide: http://docs.openstack.org/high-availability-guide/content/ch-intro.html Overcloud Services Needing HA: * rabbit * native clustering mode (http://www.rabbitmq.com/clustering.html) * activation within the code (rabbit_ha_queues flag) (http://www.rabbitmq.com/ha.html#behaviour) mirrored queues * qpid * http://openstack.redhat.com/Highly_Available_Qpid_for_OpenStack * http://qpid.apache.org/books/0.7/AMQP-Messaging-Broker-CPP-Book/html/ch01s08.html * corosync * db * Xtradb cluster * percona committing to having packages for fedeora/ubuntu/suse/centos * api instances * virtual IP - keepalived + haproxy * glance store * swift - mature, part of openstack - pending confirmation of * 1 server works * 2 servers also * or first patch that comes in * heat-engine * not stateless - failures will interrupt stack /create/update/delete * run 2 of * neutron * dhcp - set number scheduled to some N. * what about stale data? - file a bug: https://bugs.launchpad.net/neutron/+bug/1248048 * L2agent - per hypervisor - localised failures * L3agent - spof - no distributed router * routers - detect failure and reschedule networks to other routers * ideally neutron does this * metadata agent * stateless * check it runs on the network node * if runs on network node - need to schedule to new places as needed * nova-compute * nothing to do. * cinder-volume * e.g. ceph/glusterfs/3par ? * cinder bug - requests go to the originating cinder-volume even when the real data is available * horizon * N-node memcache ? * Run N copies. * virtual IP + haproxy * nova scheduler * run N copies * nova conductor * run N copies * Or just eliminate the conductor (use_local = True) General solution topics: -- load balancer strategy -- state/data preservation/migration -- load balancer as a service Actions