RDO Community News

See also blogs.rdoproject.org

RDO blogs, week of Feb 13

Here's what RDO enthusiasts have been blogging about in the last few weeks. If you blog about RDO, please let me know (rbowen@redhat.com) so I can add you to my list.

TripleO: Debugging Overcloud Deployment Failure by bregman

You run ‘openstack overcloud deploy’ and after a couple of minutes you find out it failed and if that’s not enough, then you open the deployment log just to find a very (very!) long output that doesn’t give you an clue as to why the deployment failed. In the following sections we’ll see how can […]

Read more at http://tm3.org/dv

RDO @ DevConf by Rich Bowen

It's been a very busy few weeks in the RDO travel schedule, and we wanted to share some photos with you from RDO's booth at DevConf.cz.

Read more at http://tm3.org/dw

The surprisingly complicated world of disk image sizes by Daniel Berrange

When managing virtual machines one of the key tasks is to understand the utilization of resources being consumed, whether RAM, CPU, network or storage. This post will examine different aspects of managing storage when using file based disk images, as opposed to block storage. When provisioning a virtual machine the tenant user will have an idea of the amount of storage they wish the guest operating system to see for their virtual disks. This is the easy part. It is simply a matter of telling ‘qemu-img’ (or a similar tool) ’40GB’ and it will create a virtual disk image that is visible to the guest OS as a 40GB volume. The virtualization host administrator, however, doesn’t particularly care about what size the guest OS sees. They are instead interested in how much space is (or will be) consumed in the host filesystem storing the image. With this in mind, there are four key figures to consider when managing storage:

Read more at http://tm3.org/dx

Project Leader by rbowen

I was recently asked to write something about the project that I work on – RDO – and one of the questions that was asked was:

Read more at http://tm3.org/dy

os_type property for Windows images on KVM by Tim Bell

The OpenStack images have a long list of properties which can set to describe the image meta data. The full list is described in the documentation. This blog reviews some of these settings for Windows guests running on KVM, in particular for Windows 7 and Windows 2008R2.

Read more at http://tm3.org/dz

Commenting out XML snippets in libvirt guest config by stashing it as metadata by Daniel Berrange

Libvirt uses XML as the format for configuring objects it manages, including virtual machines. Sometimes when debugging / developing it is desirable to comment out sections of the virtual machine configuration to test some idea. For example, one might want to temporarily remove a secondary disk. It is not always desirable to just delete the configuration entirely, as it may need to be re-added immediately after. XML has support for comments which one might try to use to achieve this. Using comments in XML fed into libvirt, however, will result in an unwelcome suprise – the commented out text is thrown into /dev/null by libvirt.

Read more at http://tm3.org/d-

Videos from the CentOS Dojo, Brussels, 2017 by Rich Bowen

Last Friday in Brussels, CentOS enthusiasts gathered for the annual CentOS Dojo, right before FOSDEM.

Read more at http://tm3.org/dp

FOSDEM Day 0 - CentOS Dojo by Rich Bowen

FOSDEM starts tomorrow in Brussels, but there's always a number of events the day before.

Read more at http://tm3.org/dq

Gnocchi 3.1 unleashed by Julien Danjou

It's always difficult to know when to release, and we really wanted to do it earlier. But it seems that each week more awesome work was being done in Gnocchi, so we kept delaying it while having no pressure to push it out.

Read more at http://tm3.org/dr

Testing RDO with Tempest: new features in Ocata by ltoscano

The release of Ocata, with its shorter release cycle, is close and it is time to start a broader testing (even if one could argue that it is always time for testing!).

Read more at http://tm3.org/ds

Barely Functional Keystone Deployment with Docker by Adam Young

My eventual goal is to deploy Keystone using Kubernetes. However, I want to understand things from the lowest level on up. Since Kubernetes will be driving Docker for my deployment, I wanted to get things working for a single node Docker deployment before I move on to Kubernetes. As such, you’ll notice I took a few short cuts. Mostly, these involve configuration changes. Since I will need to use Kubernetes for deployment and configuration, I’ll postpone doing it right until I get to that layer. With that caveat, let’s begin.

Read more at http://tm3.org/dt

View article »

RDO @ DevConf

It's been a very busy few weeks in the RDO travel schedule, and we wanted to share some photos with you from RDO's booth at DevConf.cz.

RDO @ DevConf

Led by Eliska Malikova, and supported by our team of RDO engineers, we provided information about RDO and OpenStack, as well as a few impromptu musical performances.

RDO @ DevConf

RDO engineers spun up a small RDO cloud, and later in the day, the people from the Manage IQ booth next door set up an instance of their software to manage that cloud, showing that RDO and Manage IQ are better together.

You can see the full album of photos on Flickr.

If you have photos or stories from DevConf, please share them with us on rdo-list. Thanks!

View article »

Videos from the CentOS Dojo, Brussels, 2017

Last Friday in Brussels, CentOS enthusiasts gathered for the annual CentOS Dojo, right before FOSDEM.

While there was no official videographer for the event, I set up my video camera in the talks that I attended, and so have video of five of the sessions.

First, I attended the session covering RDO CI in the CentOS build management system. I was a little late to this talk, so it is missing the first few minutes.

Next, I attended an introduction to Foreman, by Ewoud Kohl van Wijngaarden

Spiros Trigazis spoke about CERN's OpenStack cloud. Unfortunately, the audio is not great in this one.

Nicolas Planel, Sylvain Afchain and Sylvain Baubeau spoke about the Skydive network analyzer tool.

Finally, there was a demo of Cockpit - the Linux management console by Stef Walter. The lighting is a little weird in here, but you can see the screen even when you can't see Stef.

View article »

FOSDEM Day 0 - CentOS Dojo

FOSDEM starts tomorrow in Brussels, but there's always a number of events the day before.

This year, several speakers from the RDO community are participating in the CentOS Dojo ahead of the main event tomorrow.

  • Haïkel Guémar, Matthieu Huin - CI in the cloud - How RDO uses OpenStack infra tools for packaging
  • Spyros Trigazis - OpenStack @ CERN: Status update
  • Nicolas Planel, Sylvain Afchain, Sylvain Baubeau - Skydive - a real time network analyzer

Of course, since RDO works so closely with the CentOS CI infrastructure, all of the other content is relevant too. We're looking forward to learning about the various aspects of the CentOS project, and strengthening our bonds between the two communities, today, and in the coming years.

Here's KB addressing the opening session (Video).

View article »

Testing RDO with Tempest: new features in Ocata

The release of Ocata, with its shorter release cycle, is close and it is time to start a broader testing (even if one could argue that it is always time for testing!).

One of the core pieces for testing the cloud is the Tempest.

Tempest in RDO

Current status

The status up to the Newton is well described in few blog posts, either from packages or from git. In short, RDO used a forked repository, which regularly received all the changes from the official Tempest. The main reason for this was a configuration script which auto-discovers the features of the cloud (with some hints based on the version) and creates a valid tempest.conf.

Changes in Ocata

The auto-configuration script was decoupled from the internal Tempest and moved to the new python-tempestconf repository, thanks to the work of Martin Kopec, Chandan Kumar and Daniel Mellado.

This means less burden work required (no need to keep the fork) but also simplifies a bit the steps required to initialize and run Tempest tests, close to the process documented by Tempest upstream.

New configuration steps

Configure the RDO repositories, then install the required packages:

$ yum -y install openstack-tempest

python-tempestconf will be installed as well, as new dependency of openstack-tempest.

Now source the admin credentials, initialize tempest and run the discovery tool:

$ source </path/to/>keystonerc_admin
$ tempest init testingdir
$ cd testingdir
$ discover-tempest-config --debug identity.uri $OS_AUTH_URL \
      identity.admin_password  $OS_PASSWORD --create

discover-tempest-config is the new name of the old config_tempest.py script and it accepts the same parameters.

And that's it! Now it is possible to lists and run the tests a usual:

$ tempest run

For more details, see the the upstream documentation of tempest run. As this is a wrapper to testr, ostestr (and direct calls to testr) works as before, even if the usage of tempest run and its filtering features is highly recommended.

Tempest plugins

Current status

Tempest is really composed by two big parts: a library (with an increasing number of stable APIs) and a set of tests. With the introduction of Tempest Plugins the scope of the tests included in tempest.git was limited to the core components (right now Keystone, Nova, Neutron, Cinder, Glance and Swift).

The tests for other projects, as well as the advanced tests for the core components, have been moved to separate repositories. In most of the cases they have been added to the same repository of the main project. This introduces a complication when the tests are split in a separate RPM subpackage, as it is the case in RDO: the entry point for the tests for a component foo is always installed as part of the base subpackage for foo (usually python-foo), but the corresponding code is not (the python-foo-tests is not required). Running any tempest command, the entry points are found but the code could not be there, leading to errors.

The script install_test_packages.py provided in the openstack-tempest RPM could discover the missing entry points and install the required package, but that was clearly a workaround with a maintainance burden.

Changes in Ocata

Thanks again to the work by Chandan Kumar, the packaging was fixed to prevent this problem by automagically tuning the entry points in the generated packages. The interesting technical details are described in a past blog post.

So no more obscure errors due to missing packages while using Tempest, but you may want to check which packages containing tests are really installed if you really want to maximize the testing against your cloud.

View article »

RDO at FOSDEM this week

This week, RDO will have a presence at the FOSDEM conference, in Brussels, Belgium.

First, on Friday, there will be a number of RDO-related presentations at the CentOS Dojo which will be held at the Marriott Grand Place. Registration is free but there are a very limited number of spaces available, so register soon to secure your spot.

FOSDEM_Exhibits_day2 (17 of 38).jpg

On Saturday and Sunday, RDO will be sharing space with CentOS in the stands area, along with dozens of other Free/Open Source organizations, projects, and foundations. We'll also be helping to staff the OpenStack stand.

If you're going to be at FOSDEM, and want to help us staff the OpenStack stand, have a look at the schedule and sign up.

View article »

Last week's blog posts

Here's what RDO enthusiasts have been blogging about in the past few weeks. If you blog about RDO and you're not on this list, please let me know so I can add you to the list.

Standalone Cinder: The definitive SDS by geguileo

Are you looking for the best Software Defined Storage in the market? Look no further, Standalone Cinder is here! Let’s have an overview of the Standalone Cinder service, see some specific configurations, and find out how to make requests with no other OpenStack service is deployed. Cinder Until not so long ago Cinder was always […]

Read more at http://tm3.org/dh

ANNOUNCE: new libvirt console proxy project by Daniel Berrange

This post is to announce the existence of a new small project to provide a websockets proxy explicitly targeting virtual machines serial consoles and SPICE/VNC graphical displays.

Read more at http://tm3.org/di

OpenStack and services for BigData applications by Carlos Camacho

Yesterday I had the opportunity of presenting together with Daniel Mellado a brief talk about OpenStack and it’s integration with services to support Big Data tools (OpenStack Sahara).

Read more at http://tm3.org/dj

OpenStack Use Cases – New Analyst Papers and Webinar Now Available by Peter Pawelski, Product Marketing Manager, Red Hat OpenStack Platform

As the OpenStack market continues to mature, some organizations have made the move and put OpenStack projects into production. They have done this in a variety of ways for a variety of reasons. However, other organizations have waited to see what these first-movers are doing with it and whether or not they are successful before exploring for themselves.  

Read more at http://tm3.org/dk

We'll be at DevConf this weekend! by Rich Bowen

Come see us this weekend at DevConf.cz,in Brno, Czechia. Like last year, we'll be just inside the entrance, with stickers, command-line cheat sheets, RDO tshirts, and people to answer all of your RDO and OpenStack questions.

Read more at http://tm3.org/dl

What are Clouds? by Zane Bitter

Like many in the community, I am often called upon to explain what OpenStack is to somebody completely unfamiliar with it. Usually this goes one of two ways: they turn out to be familiar enough with cloud computing to quickly grasp it by analogy, or their eyes glaze over at the mention of the words ‘cloud computing’ and no further explanation is sought or offered. When faced with someone who is persistently curious but not an industry insider, you immediately know you’re in trouble.

Read more at http://tm3.org/dm

View article »

Last week's blog posts

Making sure your Gerrit changes aren't broken by Lars Kellogg-Stedman

It's a bit of an embarrassment when you submit a review to Gerrit only to have it fail CI checks immediately because of something as simple as a syntax error or pep8 failure that you should have caught yourself before submitting…but you forgot to run your validations before submitting the change.

Read more at http://tm3.org/de

On communities: Trading off our values… Sometimes by Flavio Percoco

Not long ago I wrote about how much emotions matter in every community. In that post I explained the importance of emotions, how they affect our work and why I believe they are relevant for pretty much everything we do. Emotions matter is a post quite focused on how we can affect, with our actions, other people's emotional state.

Read more at http://tm3.org/df

9 tips to properly configure your OpenStack Instance by Marko Myllynen

In OpenStack jargon, an Instance is a Virtual Machine, the guest workload. It boots from an operating system image, and it is configured with a certain amount of CPU, RAM and disk space, amongst other parameters such as networking or security settings.

Read more at http://tm3.org/dg

Writing RPM macro for OpenStack by chandankumar

RPM macro is a short string, always prefixed by % and generally surrounded by curly brackets ({}) which RPM will convert to a different and usually longer string. Some macros can take arguments and some can be quite complex.

Read more at http://tm3.org/dc

TripleO deep dive session #7 (Undercloud - TripleO UI) by Carlos Camacho

This is the seven release of the TripleO “Deep Dive” sessions

Read more at http://tm3.org/db

Installing the TripleO UI by Carlos Camacho

This is a brief recipe to use or install TripleO UI in the Undercloud.

Read more at http://tm3.org/dd

View article »

Writing RPM macro for OpenStack

RPM macro is a short string, always prefixed by % and generally surrounded by curly brackets ({}) which RPM will convert to a different and usually longer string. Some macros can take arguments and some can be quite complex.

In RHEL, CentOS and Fedora, macros are provided by rpm package and from redhat-rpm-config.

In RDO, OpenStack macros are provided by openstack-macros which comes from upstream rpm-packaging project.

You can find list of all macros under /usr/lib/rpm/macros.d/ directory.

To see the list of all available macros on your system:

$ rpm --showrc

For example: %{_bindir} is a rpm-macro which points to the binary directory where executables are usually stored.

To evaluate an rpm macro:

$ rpm --eval %{_bindir}

%py_build is the commonly used rpm-macro in RDO OpenStack packages which points to python setup.py build process.

$ rpm --eval %py_build

Motivation behind writing a new RPM macro for OpenStack packages

Currently, Tempest provides an external test plugin interface which enables anyone to integrate an external test suite as a part of Tempest run and each service Tempest plugin has an entrypoint defined in setup.cfg through which tempest discovers it and list the Tempest plugins. For example:

tempest.test_plugins =
    heat_tests = heat_integrationtests.plugin:HeatTempestPlugin

In RDO OpenStack services RPM packages, In-tree Tempest plugins packages are provided by openstack-{service}-tests subpackage but the tempest plugin entrypoint is provided by the main package openstack-%{service}. So once you have a working OpenStack environment with Tempest installed having no test subpackage installed. Then we tried to run tempest commands you would have encountered "No module heat_integrationtests.plugin found" and you end up installing a hell lot of packages to fix this. The basic reason for the above error is tempest plugin entry point is installed by main OpenStack package but files pointing to entrypoint are not found.

To fix the above issue we have decided to separate out the tempest plugin entrypoint from the main package and move it to openstack-{service}-tests subpackage during rpmbuild process by creating a fake tempest plugin entry point for all RDO services packages. Since it is a massive and similar change affecting all OpenStack services packages. So, I have created %py2_entrypoint macro which is available in OpenStack Ocata release.

Here is the macro definition of %py2_entrypoint:

# Create a fake tempest plugin entry point which will
# resides under %{python2_sitelib}/%{service}_tests.egg-info.
# The prefix is %py2_entrypoint %{modulename} %{service}
# where service is the name of the openstack-service or the modulename
# It should used under %install section
# the generated %{python2_sitelib}/%{service}_tests.egg-info
# will go under %files section of tempest plugin subpackage
# Example: %py2_entrypoint %{modulename} %{service}
# In most of the cases %{service} is same as %{modulename}
# but in case of neutron plugins it is different
# like servicename is neutron-lbaas and modulename is neutron_lbass
%py2_entrypoint() \
egg_path=%{buildroot}%{python2_sitelib}/%{1}-*.egg-info \
tempest_egg_path=%{buildroot}%{python2_sitelib}/%{1}_tests.egg-info \
mkdir $tempest_egg_path \
grep "tempest\\|Tempest" %{1}.egg-info/entry_points.txt >$tempest_egg_path/entry_points.txt \
sed -i "/tempest\\|Tempest/d" $egg_path/entry_points.txt \
cp -r $egg_path/PKG-INFO $tempest_egg_path \
sed -i "s/%{2}/%{1}_tests/g" $tempest_egg_path/PKG-INFO \
%nil

Here is the list of tempest-plugin-entrypoint reviews.

Some learning from above macro:

[1.] You can use the shell script or Lua language to write macros.

[2.] %define <macroname> is used to define a macro in spec file or you can directly place the macro in /usr/lib/rpm/macros.d/macros.openstack-rdo to consume it using rpmbuild process.

[3.] use %nil to showcase the end of the macro.

[4.] use %{1} to %{6} to pass variables in macros.

Above is a temporary solution. We are working upstream to separate out tempest plugins from OpenStack project to a new repo for easier management and packaging in Pike release:https://review.openstack.org/#/c/405416/.

Thanks to Daniel, Alan Haikel and many others on #rdo channel for getting the work done. It was a great learning experience.

View article »