OpenStack
Ceilometer
Tong Li (IBM)
Brad Topol (IBM)
Agenda
What is Ceilometer?
Brief history of Ceilometer
Meters of Ceilometer
Workflow
Ceilometer Architect & Building Blocks
Collecting Process
Pipeline
Transformer
Publisher
Collector
Data store drivers
Alarm
Dispatcher
Ceilometer ReSTful API
Ceilometer Python Client
Ceilometer in Devstack
Inter-Component Security & High Availability
Who are the users of OpenStack Ceilometer
Ceilometer Road Map
Q & A (Demo maybe)
Who are we?
Tong Li
IBM senior software engineer
Twitter @email4tong
Email: litong01@us.ibm.com
Brad Topol
IBM distingushed engineer
Twitter @bradtopol
Email: btopol@us.ibm.com
What is Ceilometer?
OpenStack measurement project
Julien Danjou Ceilometer PTL
A system that keeps track of what is happening in
OpenStack clusters. It was originally created for
billing but it has been evolved into an all purpose
measurement repository. It can be extended for
purposes such as performance report,
troubleshooting, capacity analytics, smart
scheduling, optimization, ultimately BIG DATA etc.
Brief History of Ceilometer
Before year 2012 billing has been left out of OpenStack core so far
as it was not the primary problem and was non-trivial, yet almost
every OpenStack deployment needs a way to track usage
information
Started in May 2012 by developers from Canonical, DreamHost,
RedHat, Dell, Intel, AT&T, etc.
Targeted to do metering, rating, billing
Grizzly objective: The project aimed to become the infrastructure to
collect measurements within OpenStack so that no two agents
would need to be written to collect the same data.
Havana objective: remains the same as Grizzly objective plus
become an integrated OpenStack project.
Current PTL: Julien Danjou
91 contributors, 7 core reviewers
Meters of Ceilometer
(from Nova)
Meters of Ceilometer
(from Neutron)
Additional measurements:
http://docs.openstack.org/developer/ceilometer/measurements.html
Workflow
Collect
Transform
Publish
Store
Read
Collect interesting info from OpenStack components
(or any other components), create meters (or samples)
Transform meters into other meters if needed
Publish meters to any destination (including
Ceilometer itself)
Store received meters into data store
Read data via Ceilometer REST API
Ceilometer Architecture
Collecting
Notification bus (AMQP)
Nova
API
queue name: notification.info
Neutron
API
Cinder
API
Glance
API
Swift
API
Polling
Notification Handlers, Central Agents
Publishing
pipeline
Publishing
New in
Grizzly
!
Pipeline
Ceilometer
collector & agent
Meter
Transformer
Transformer
Transformer
Publisher
Publisher
Publisher
Pipeline: a set of transformers mutating meters into
something that publishers know how to send to external
systems.
Configure pipelines @pipeline.yaml
Receiver
Receiver
Receiver
New in
Grizzly
!
Transformer
Meter
Name=cpu.time
Value=5
Timestamp=T
Meter
Name=cpu.time
Value=20
Timestamp=T+1
Transformer
Meter
Name=cpu.time
Value=80
Timestamp=T+2
Transform meters into new meters!
Meter
Name=cpu.percentage
Value=9
Timestamp=T+2
New in
Grizzly
!
Publisher
AMQP
signed message
Meter
Ceilometer
RPC
Publisher
Ceilometer
Message queue
Other
Publisher
External system
Billing, monitoring,
alarming, statistics,
capacity planning
Data Store
Message Queue
ceilometer.collector.metering
Meter
Dispatchers
SQL
Database
Default backend
HBase
MongoDB
Database
New in
Grizzly
New in
Havana
Multiple backend choices
IBM DB2
Alarm
New in
Havana
Both alarm evaluator and alarm notifier are
Ceilometer services
1. Alarm evaluator uses Ceilomter public APIs to
pull meters to produce alarms according to
alarm definition, then place the alarm
message onto the alarm queue
2. Alarm notifier read the alarm queue then
inform external systems (log, rest, test)
Dispatchers
New in
Havana
Multiple dispatchers support allows Ceilometer to be extended easily with many different outlet
for all meters coming into Ceilometer. Dispatchers can be easily developed and deployed
to add more features. For example, sending email, text messages.
1. Dispatcher only needs to implement one method.
2. By default, database dispatcher is used even if the configuration file has no trace of any
dispatchers.
3. To configure multiple dispatchers:
[ceilometer.dispatcher]
file = ceilometer.collector.dispatcher.file:FileDispatcher
database = ceilometer.collector.dispatcher.database:DatabaseDispatcher
Ceilometer ReSTful API
request
Ceilometer API
Consumers
Billing, monitoring,
alarming, statistics,
capacity planning
Ceilometer API
HTTP
REST
w/ JSON &
XML
Storage abstraction
layer
Raw events:
GET /v2/meters/vcpus
Statistics (sum, average, min, max):
GET /v2/meters/vcpus/statistics
New in
Grizzly
!
Filter:
GET /v2/meters/vcpus?q.field=project_id&q.value=foobar&q.op=eq
Group statistics by period:
GET /v2/meters/vcpus/statistics?period=3600
Database
Ceilometer ReSTful API
response
Response can be in JSON or XML
Raw events:
/v2/meters/storage.object.size
Statistics (sum, average, min, max):
/v2/meters/storage.api.request/statistics
Ceilometer Python Client
Ceilometer in Devstack
devstack localrc
Enable the ceilometer metering services
enable_service ceilometer-acompute,ceilometer-acentral,ceilometer-collector
Enable the ceilometer alarming services
enable_service ceilometer-alarm-evaluator,ceilometer-alarm-notifier
Enable the ceilometer api services
enable_service ceilometer-api
Inter-Component Security
Ceilometer High Availability
Who uses Ceilometer?
https://etherpad.openstack.org/p/CeilometerFeeedback
Roadmap
Grizzly
Incubated Project
Integration
with Horizon
Agents for other
components
o Swift
o Ceph?
o Nicira?
SQLAlchemy storage
driver
Multi-Publisher
API v2
o User accessible API
o More aggregation
o Multi-dimension
Havana
Integrated Project
Integration with Horizon
Publishing meters to other
systems
Enhance SQL driver
Alarming
Integration with Heat
Deprecating APIv1
Completing APIv2
Move publishing part to
Oslo and other projects
Tighter integration with
Nova
Nova-scheduler integration
Icehouse
OpenStack Design
Summit in Hong Kong
Questions?
http://launchpad.net/ceilometer
http://docs.openstack.org/developer/ceilometer
http://wiki.openstack.org/ceilometer
Freenode:
Mailing List:
#openstack-metering
openstack-dev [ceilometer]
