RDO Community News

See also blogs.rdoproject.org

Videos from the CentOS Dojo, Brussels, 2017

Last Friday in Brussels, CentOS enthusiasts gathered for the annual CentOS Dojo, right before FOSDEM.

While there was no official videographer for the event, I set up my video camera in the talks that I attended, and so have video of five of the sessions.

First, I attended the session covering RDO CI in the CentOS build management system. I was a little late to this talk, so it is missing the first few minutes.

Next, I attended an introduction to Foreman, by Ewoud Kohl van Wijngaarden

Spiros Trigazis spoke about CERN's OpenStack cloud. Unfortunately, the audio is not great in this one.

Nicolas Planel, Sylvain Afchain and Sylvain Baubeau spoke about the Skydive network analyzer tool.

Finally, there was a demo of Cockpit - the Linux management console by Stef Walter. The lighting is a little weird in here, but you can see the screen even when you can't see Stef.

View article »

FOSDEM Day 0 - CentOS Dojo

FOSDEM starts tomorrow in Brussels, but there's always a number of events the day before.

This year, several speakers from the RDO community are participating in the CentOS Dojo ahead of the main event tomorrow.

  • Haïkel Guémar, Matthieu Huin - CI in the cloud - How RDO uses OpenStack infra tools for packaging
  • Spyros Trigazis - OpenStack @ CERN: Status update
  • Nicolas Planel, Sylvain Afchain, Sylvain Baubeau - Skydive - a real time network analyzer

Of course, since RDO works so closely with the CentOS CI infrastructure, all of the other content is relevant too. We're looking forward to learning about the various aspects of the CentOS project, and strengthening our bonds between the two communities, today, and in the coming years.

Here's KB addressing the opening session (Video).

View article »

Testing RDO with Tempest: new features in Ocata

The release of Ocata, with its shorter release cycle, is close and it is time to start a broader testing (even if one could argue that it is always time for testing!).

One of the core pieces for testing the cloud is the Tempest.

Tempest in RDO

Current status

The status up to the Newton is well described in few blog posts, either from packages or from git. In short, RDO used a forked repository, which regularly received all the changes from the official Tempest. The main reason for this was a configuration script which auto-discovers the features of the cloud (with some hints based on the version) and creates a valid tempest.conf.

Changes in Ocata

The auto-configuration script was decoupled from the internal Tempest and moved to the new python-tempestconf repository, thanks to the work of Martin Kopec, Chandan Kumar and Daniel Mellado.

This means less burden work required (no need to keep the fork) but also simplifies a bit the steps required to initialize and run Tempest tests, close to the process documented by Tempest upstream.

New configuration steps

Configure the RDO repositories, then install the required packages:

$ yum -y install openstack-tempest

python-tempestconf will be installed as well, as new dependency of openstack-tempest.

Now source the admin credentials, initialize tempest and run the discovery tool:

$ source </path/to/>keystonerc_admin
$ tempest init testingdir
$ cd testingdir
$ discover-tempest-config --debug identity.uri $OS_AUTH_URL \
      identity.admin_password  $OS_PASSWORD --create

discover-tempest-config is the new name of the old config_tempest.py script and it accepts the same parameters.

And that's it! Now it is possible to lists and run the tests a usual:

$ tempest run

For more details, see the the upstream documentation of tempest run. As this is a wrapper to testr, ostestr (and direct calls to testr) works as before, even if the usage of tempest run and its filtering features is highly recommended.

Tempest plugins

Current status

Tempest is really composed by two big parts: a library (with an increasing number of stable APIs) and a set of tests. With the introduction of Tempest Plugins the scope of the tests included in tempest.git was limited to the core components (right now Keystone, Nova, Neutron, Cinder, Glance and Swift).

The tests for other projects, as well as the advanced tests for the core components, have been moved to separate repositories. In most of the cases they have been added to the same repository of the main project. This introduces a complication when the tests are split in a separate RPM subpackage, as it is the case in RDO: the entry point for the tests for a component foo is always installed as part of the base subpackage for foo (usually python-foo), but the corresponding code is not (the python-foo-tests is not required). Running any tempest command, the entry points are found but the code could not be there, leading to errors.

The script install_test_packages.py provided in the openstack-tempest RPM could discover the missing entry points and install the required package, but that was clearly a workaround with a maintainance burden.

Changes in Ocata

Thanks again to the work by Chandan Kumar, the packaging was fixed to prevent this problem by automagically tuning the entry points in the generated packages. The interesting technical details are described in a past blog post.

So no more obscure errors due to missing packages while using Tempest, but you may want to check which packages containing tests are really installed if you really want to maximize the testing against your cloud.

View article »

RDO at FOSDEM this week

This week, RDO will have a presence at the FOSDEM conference, in Brussels, Belgium.

First, on Friday, there will be a number of RDO-related presentations at the CentOS Dojo which will be held at the Marriott Grand Place. Registration is free but there are a very limited number of spaces available, so register soon to secure your spot.

FOSDEM_Exhibits_day2 (17 of 38).jpg

On Saturday and Sunday, RDO will be sharing space with CentOS in the stands area, along with dozens of other Free/Open Source organizations, projects, and foundations. We'll also be helping to staff the OpenStack stand.

If you're going to be at FOSDEM, and want to help us staff the OpenStack stand, have a look at the schedule and sign up.

View article »

Last week's blog posts

Here's what RDO enthusiasts have been blogging about in the past few weeks. If you blog about RDO and you're not on this list, please let me know so I can add you to the list.

Standalone Cinder: The definitive SDS by geguileo

Are you looking for the best Software Defined Storage in the market? Look no further, Standalone Cinder is here! Let’s have an overview of the Standalone Cinder service, see some specific configurations, and find out how to make requests with no other OpenStack service is deployed. Cinder Until not so long ago Cinder was always […]

Read more at http://tm3.org/dh

ANNOUNCE: new libvirt console proxy project by Daniel Berrange

This post is to announce the existence of a new small project to provide a websockets proxy explicitly targeting virtual machines serial consoles and SPICE/VNC graphical displays.

Read more at http://tm3.org/di

OpenStack and services for BigData applications by Carlos Camacho

Yesterday I had the opportunity of presenting together with Daniel Mellado a brief talk about OpenStack and it’s integration with services to support Big Data tools (OpenStack Sahara).

Read more at http://tm3.org/dj

OpenStack Use Cases – New Analyst Papers and Webinar Now Available by Peter Pawelski, Product Marketing Manager, Red Hat OpenStack Platform

As the OpenStack market continues to mature, some organizations have made the move and put OpenStack projects into production. They have done this in a variety of ways for a variety of reasons. However, other organizations have waited to see what these first-movers are doing with it and whether or not they are successful before exploring for themselves.  

Read more at http://tm3.org/dk

We'll be at DevConf this weekend! by Rich Bowen

Come see us this weekend at DevConf.cz,in Brno, Czechia. Like last year, we'll be just inside the entrance, with stickers, command-line cheat sheets, RDO tshirts, and people to answer all of your RDO and OpenStack questions.

Read more at http://tm3.org/dl

What are Clouds? by Zane Bitter

Like many in the community, I am often called upon to explain what OpenStack is to somebody completely unfamiliar with it. Usually this goes one of two ways: they turn out to be familiar enough with cloud computing to quickly grasp it by analogy, or their eyes glaze over at the mention of the words ‘cloud computing’ and no further explanation is sought or offered. When faced with someone who is persistently curious but not an industry insider, you immediately know you’re in trouble.

Read more at http://tm3.org/dm

View article »

Last week's blog posts

Making sure your Gerrit changes aren't broken by Lars Kellogg-Stedman

It's a bit of an embarrassment when you submit a review to Gerrit only to have it fail CI checks immediately because of something as simple as a syntax error or pep8 failure that you should have caught yourself before submitting…but you forgot to run your validations before submitting the change.

Read more at http://tm3.org/de

On communities: Trading off our values… Sometimes by Flavio Percoco

Not long ago I wrote about how much emotions matter in every community. In that post I explained the importance of emotions, how they affect our work and why I believe they are relevant for pretty much everything we do. Emotions matter is a post quite focused on how we can affect, with our actions, other people's emotional state.

Read more at http://tm3.org/df

9 tips to properly configure your OpenStack Instance by Marko Myllynen

In OpenStack jargon, an Instance is a Virtual Machine, the guest workload. It boots from an operating system image, and it is configured with a certain amount of CPU, RAM and disk space, amongst other parameters such as networking or security settings.

Read more at http://tm3.org/dg

Writing RPM macro for OpenStack by chandankumar

RPM macro is a short string, always prefixed by % and generally surrounded by curly brackets ({}) which RPM will convert to a different and usually longer string. Some macros can take arguments and some can be quite complex.

Read more at http://tm3.org/dc

TripleO deep dive session #7 (Undercloud - TripleO UI) by Carlos Camacho

This is the seven release of the TripleO “Deep Dive” sessions

Read more at http://tm3.org/db

Installing the TripleO UI by Carlos Camacho

This is a brief recipe to use or install TripleO UI in the Undercloud.

Read more at http://tm3.org/dd

View article »

Writing RPM macro for OpenStack

RPM macro is a short string, always prefixed by % and generally surrounded by curly brackets ({}) which RPM will convert to a different and usually longer string. Some macros can take arguments and some can be quite complex.

In RHEL, CentOS and Fedora, macros are provided by rpm package and from redhat-rpm-config.

In RDO, OpenStack macros are provided by openstack-macros which comes from upstream rpm-packaging project.

You can find list of all macros under /usr/lib/rpm/macros.d/ directory.

To see the list of all available macros on your system:

$ rpm --showrc

For example: %{_bindir} is a rpm-macro which points to the binary directory where executables are usually stored.

To evaluate an rpm macro:

$ rpm --eval %{_bindir}

%py_build is the commonly used rpm-macro in RDO OpenStack packages which points to python setup.py build process.

$ rpm --eval %py_build

Motivation behind writing a new RPM macro for OpenStack packages

Currently, Tempest provides an external test plugin interface which enables anyone to integrate an external test suite as a part of Tempest run and each service Tempest plugin has an entrypoint defined in setup.cfg through which tempest discovers it and list the Tempest plugins. For example:

tempest.test_plugins =
    heat_tests = heat_integrationtests.plugin:HeatTempestPlugin

In RDO OpenStack services RPM packages, In-tree Tempest plugins packages are provided by openstack-{service}-tests subpackage but the tempest plugin entrypoint is provided by the main package openstack-%{service}. So once you have a working OpenStack environment with Tempest installed having no test subpackage installed. Then we tried to run tempest commands you would have encountered "No module heat_integrationtests.plugin found" and you end up installing a hell lot of packages to fix this. The basic reason for the above error is tempest plugin entry point is installed by main OpenStack package but files pointing to entrypoint are not found.

To fix the above issue we have decided to separate out the tempest plugin entrypoint from the main package and move it to openstack-{service}-tests subpackage during rpmbuild process by creating a fake tempest plugin entry point for all RDO services packages. Since it is a massive and similar change affecting all OpenStack services packages. So, I have created %py2_entrypoint macro which is available in OpenStack Ocata release.

Here is the macro definition of %py2_entrypoint:

# Create a fake tempest plugin entry point which will
# resides under %{python2_sitelib}/%{service}_tests.egg-info.
# The prefix is %py2_entrypoint %{modulename} %{service}
# where service is the name of the openstack-service or the modulename
# It should used under %install section
# the generated %{python2_sitelib}/%{service}_tests.egg-info
# will go under %files section of tempest plugin subpackage
# Example: %py2_entrypoint %{modulename} %{service}
# In most of the cases %{service} is same as %{modulename}
# but in case of neutron plugins it is different
# like servicename is neutron-lbaas and modulename is neutron_lbass
%py2_entrypoint() \
egg_path=%{buildroot}%{python2_sitelib}/%{1}-*.egg-info \
tempest_egg_path=%{buildroot}%{python2_sitelib}/%{1}_tests.egg-info \
mkdir $tempest_egg_path \
grep "tempest\\|Tempest" %{1}.egg-info/entry_points.txt >$tempest_egg_path/entry_points.txt \
sed -i "/tempest\\|Tempest/d" $egg_path/entry_points.txt \
cp -r $egg_path/PKG-INFO $tempest_egg_path \
sed -i "s/%{2}/%{1}_tests/g" $tempest_egg_path/PKG-INFO \
%nil

Here is the list of tempest-plugin-entrypoint reviews.

Some learning from above macro:

[1.] You can use the shell script or Lua language to write macros.

[2.] %define <macroname> is used to define a macro in spec file or you can directly place the macro in /usr/lib/rpm/macros.d/macros.openstack-rdo to consume it using rpmbuild process.

[3.] use %nil to showcase the end of the macro.

[4.] use %{1} to %{6} to pass variables in macros.

Above is a temporary solution. We are working upstream to separate out tempest plugins from OpenStack project to a new repo for easier management and packaging in Pike release:https://review.openstack.org/#/c/405416/.

Thanks to Daniel, Alan Haikel and many others on #rdo channel for getting the work done. It was a great learning experience.

View article »

Recent blog posts

I've been out for a few weeks, but the blog posts from the community kept coming.

Containers on the CERN cloud by Tim Bell

We have recently made the Container-Engine-as-a-Service (Magnum) available in production at CERN as part of the CERN IT department services for the LHC experiments and other CERN communities. This gives the OpenStack cloud users Kubernetes, Mesos and Docker Swarm on demand within the accounting, quota and project permissions structures already implemented for virtual machines.We shared the latest news on the service with the CERN technical staff (link). This is the follow up on the tests presented at the OpenStack Barcelona (link) and covered in the blog from IBM.

Read more at http://tm3.org/d6

ANNOUNCE: New libvirt project Go XML parser model by Daniel Berrange

Shortly before christmas, I announced the availability of new Go bindings for the libvirt API. This post announces a companion package for dealing with XML parsing/formatting in Go. The master repository is available on the libvirt GIT server, but it is expected that Go projects will consume it via an import of the github mirror, since the Go ecosystem is heavilty github focused (e.g. godoc.org can’t produce docs for stuff hosted on libvirt.org git)

Read more at http://tm3.org/d7

Red Hat OpenStack Platform 10 is here! So what’s new? by Marcos Garcia - Principal Technical Marketing Manager

It’s that time of the year. We all look back at 2016, think about the good and bad things, and wish that Santa brings us the gifts we deserve. We, at Red Hat, are really proud to bring you a present for this holiday season: a new version of Red Hat OpenStack Platform, version 10 (press release and release notes). This is our best release ever, so we’ve named it our first Long Life release (up to 5 years support), and this blog post will show you why this will be the perfect gift for your private cloud project.

Read more at http://tm3.org/d8

Comparing OpenStack Neutron ML2+OVS and OVN – Control Plane by russellbryant

We have done a lot of performance testing of OVN over time, but one major thing missing has been an apples-to-apples comparison with the current OVS-based OpenStack Neutron backend (ML2+OVS).  I’ve been working with a group of people to compare the two OpenStack Neutron backends.  This is the first piece of those results: the control plane.  Later posts will discuss data plane performance.

Read more at http://tm3.org/d9

View article »

Blog posts last week

TripleO to deploy Ceph standlone by Giulio Fidente

Here is a nice Christmas present: you can use TripleO for a standalone Ceph deployment, with just a few lines of YAML. Assuming you have an undercloud ready for a new overcloud, create an environment file like the following:

Read more at http://tm3.org/d1

Printed TripleO cheatsheets for FOSDEM/DevConf (feedback needed) by Carlos Camacho

We are working preparing some cheatsheets for people jumping into TripleO.

Read more at http://tm3.org/d2

ANNOUNCE: New libvirt project Go language bindings by Daniel Berrange

I’m happy to announce that the libvirt project is now supporting Go language bindings as a primary deliverable, joining Python and Perl, as language bindings with 100% API coverage of libvirt C library. The master repository is available on the libvirt GIT server, but it is expected that Go projects will consume it via an import of the github mirror, since the Go ecosystem is heavilty github focused (e.g. godoc.org can’t produce docs for stuff hosted on libvirt.org git)

Read more at http://tm3.org/d3

**A Quick Introduction to Mistral Usage in TripleO (Newton) For developers** by jpichon

Since Newton, Mistral has become a central component to the TripleO project, handling many of the operations in the back-end. I recently gave a short crash course on Mistral, what it is and how we use it to a few people and thought it might be useful to share some of my bag of tricks here as well.

Read more at http://tm3.org/d4

Lifecycle support changes for Red Hat OpenStack Platform 10 and beyond by Peter Pawelski, Product Marketing Manager, Red Hat OpenStack Platform

During the past six years, OpenStack has evolved rapidly. The OpenStack community itself has grown to more than 60,000 strong, with support from a wide array of technology vendors across the globe. Customers are pushing OpenStack into production and starting to realize the many benefits OpenStack has been promising them.

Read more at http://tm3.org/d5

View article »