IBM Integrated Solution for System z Development (ISDz) Leveraging the principles of DevOps
Henk.van.der.wijk@nl.ibm.com
10 juli 2014
Systems of Interaction challenges Speed mismatch between faster moving Systems of Engagement and slower moving Systems of Record, delaying time to customer feedback
Four things that are different about developing applications for the mainframe environment
IBM Integrated Solution for System z Development Leveraging the principles of DevOps to deliver high quality applications
Rational Developer for the Enterprise: THE Premier Integrated Development Environment for System z & Multi-Platform Development
Cost Effective Off-Host Development and Unit Test
DevOps will dramatically increase build and test activity
Makes it affordable by offloading from System z

If you knew what I know or CloudWave – Improving services in the Cloud through collaborative adaptation
Eliot Salant
salant@il.ibm.com
IBM Haifa Research
CloudWave Project Coordinator
Where things are today
So, what’ s the problem? Hint
Evolution of software delivery
Release times (wks) -Windows vs. app
The DevOps paradigm

The CloudWave idea
CloudWave overview www.cloudwave-fp7.eu
Main project concepts
Some CloudWave Challenges
Strategy
Functional Decomposition
Terminology
Concept
CW Monitoring
Adding analytics
Living State Manager
All together now
Coordinated Adaptation
Directions for Coordinated Adaptation
Machine learning to react to enactment point triggers
Adaptation of both infrastructure and application
Determination of new enactment points

Some challenges
Multiple layers for adaptation
Coordinated adaptation challenges
Ultimate effect of adaptation actions at different levels not always clear
Sample set for machine learning
Standardizing application adaptations

Example of potential coordinated adaptation
Computations on a mobile phone vs. in Cloud
IoT devices -autonomy vs. centralized control
Feedback Drive Design
Better monitoring information and analysis to help developers
Analysis of Adaptation Engine efficiency
Feedback driven testing
Evolution of testing
Problem recreation

FDD Challenges
Effective feedback visualization
Intelligent hints to developers (analysis)
What-if analysis
In summary

DevOps.

Continuous Delivery
Agile Software Development

My Background
Continuous Integration
Scrum
Deployment scripts
Testing
DevOps
…
But: how to tie all this together?
What is it?
Keywords:
Automation
Quality
Releasing (as opposed to òjust’ deploying)
Where does it fit?
The DevOps movement
Shares common goals
Development vs Operations
Deployment automation
Age-old practice by smart developers
Loads of scripts, tools, commercial and open source products
Part of Continuous Delivery
Where does it fit?
Agile and lean software development
Potentially shippable product (Scrum)
Deliver as Fast as possible (Lean)
Many concepts in Continuous Delivery come from Lean
Release Early, Release Often
Continuous Delivery
A very good book
Explains how to design/set up your continuous delivery proces
Including CI, deployments, etc.
Explains good software development practices that are necessary for CD
Anti-Pattern: Deploying Software Manually
Needs lots of documentation with steps
That are misinterpreted, and untested in a production-like environment
Many corrections during a release
Manual testing to confirm deployment
Releases take hours instead of minutes
Frequent roll-backs
Late nights
Instead…
Automate deployments as much as possible
Scripts equal up-to-date documentation, encourage cooperation and sharing
Encapsulate expertise and are version-controlled like any other source code
Automation makes processes repeatable, testable, less boring and fast.
AP: First to production-like after Dev finished
Testers have only tested in a development environment
Production deployers have never done deployment before
All kinds of infrastructure-related problems pop up after the PL said that development was done
Lots of fighting between operations and development
Instead…
Start deploying to a production-like environment (staging) as soon as you start developing
Integrate testing, deployment and release into a standard part of your development process
If you already have good Continuous Integration, this means: add deployment/release to it (correctly)
AP: Manual Configuration of Prod Env
Configuration of an environment is a lot
Deployments to production fails after succesful deployments to other environments
Differences between nodes in production
Cannot roll back your environment
Different versions of packages
Basically: No overview, no control
Instead…
Set up your entire environment using an automated process that is stored in version control
Specifically any and all configuration – of your OS, of OS tools and middleware, and of your application
Your production environment is key – it must be reproducible in your staging and dev environments
Feedback slide
How many people
use version control for their source code?
use Continuous Integration?
have automated deployments?
do Continuous Delivery?
The Deployment Pipeline
Describes your process of getting software from version control into the hands of users.
Similar on for most projects on a high level
Inspired by òValue Stream Maps’ from Poppendieck’ s Lean Software Development book
Each step in the pipeline produces output which is taken up (pulled) by the next step when ready
A Deployment Pipeline
Also known as Continuous Integration Pipeline, build pipeline, etc.
Trade-offs
Early feedback v.s. more thorough testing
Early feedback v.s. more production-like environments
How long is a òbuild’ allowed to take?
Can it be split up?
Continuous Integration PreReqs
Version Control
A command-line automated build, outside IDE
Team agreement to do CI properly
Check in frequently
Maintain a sizeable Automated Test Suite
Keep build and test phase short (*)
Keep your workspace clean
CI how-to
Never check in on a broken build!!
When you’ re done developing:
check to see if build is running – wait to see if it succeeds
update to latest version
build everything locally, and if it passes
check your code into version control
wait til CI server succeeds (don’ t go home)
Noteworthy
Teams that are not in the same room need to have better agreements
Time zone differences need to be managed (and especially: never go home on a broken build)
Branches degrade CI usefulness
Distributed version control systems can be a pain and need to be managed very well if they are really distributed
Tools
Hudson (Jenkins)
TeamCity
Anthill Pro
Cruise Control
Bamboo
…
Questions
What languages you use?
What build tools do you use?
Multi-stage builds
After the commit stage, an automated acceptance testing stage is typically run.
Depending on your CI servers this can be a build that is automatically triggered by the completion of the commit stage build.
Some CI servers can model this nicer.
Automated acceptance tests
They typically run much longer.
Some CI benefits are lost; this is a trade-off.
Before running these, the application (and the environment) is first deployed automatically
Practices like BDD and ATDD can be used here

Parallelization can speed up things a lot
Idea: use EC2 to run tests
Manual Testing
Most applications still need manual testing
Set up a page on which the testers can instantly deploy the right version of the app and environment onto their testing environment
Releasing
Releasing to production takes more
Don’ t forget maintenance/support, monitoring, environments, change process, application configuration, external systems, logging, disaster recovery, SLA, capacity, upgrades, etc.
Agree on things as soon as you can: release strategy
Overlaps area of DevOps
How to automate?
Your own scripts (Ruby is popular)
Frameworks (Capistrano, Fabric, …)
DSLs (Puppet, Chef)
(Commercial) products
XebiaLabs DeployIt, ThoughtWorks Go
Gazillions of other tools
The provisioning stack
Continuous Deployment
Deploy all changes that pass your automated tests to production
Pro’ s and Con’ s

Only when there’ s time left
Wrap-up
Read the book – very inspiring, full of useful detail
Adjust your definition of done – it’ s only done when it’ s deployed
Grow over time: improve your testing strategy, make use of new possibilities
Practical session later?
For example, setting up an environment with Vagrant and configure it with Puppet/Chef.

?Brochure
??Four keys to your DevOps journey
Bring together apps and operations with DevOps principles
?
?Table of contents
3 DevOps defined
4 The challenge: keeping pace with a high-velocity business
5 Driving collaboration across the development lifecycle and IT operations
6 A holistic approach to DevOps
7 Get on the path to DevOps with these four keys
8 Key 1. Assess your DevOps strategy
9 Key 2. Identify the DevOps maturity of your core development and IT operations processes
11 Key 3. Adopt and implement change to improve velocity
12 Key 4. Measure progress and plan next improvement
13 Make your journey to DevOps with HPE Software Professional Services
??DevOps defined
Devops is a set of best practices and cultural changes that allow development, quality assurance, and operations to better meet customer needs. This integrated team approach facilitates communication, collaboration, and integration to manage today’s rapidly changing business demands.
Brochure
?DevOps defined
The challenge
Driving collaboration HPE approach
Four keys
HPE Software Professional Services
The challenge: keeping pace with a high-velocity business
To stay competitive, your business needs to accelerate the delivery of new software features and functionality. That’s the idea behind the agile software-development processes that are now widely used by application delivery teams to reduce delivery cycle times.
But here’s where things get harder. While agile methods are a huge step forward for application delivery teams, they alone don’t ensure the fast rollout of new software. Cycle times are also driven by IT operations teams, which historically have been a speed bump in the road to putting applications into production.
Why the delays? Operations teams want to avoid the risks that come with changes made to applications and infrastructure, such as breaching service-level agreements (SLAs), system downtime, and outages. So they approach new software releases cautiously. The result: New software releases are delayed, and business agility is lost.
DevOps helps your organization to bring together the key stakeholders (business, apps, and ops) by focusing on collaboration, automation, and monitoring, resulting in improved application release velocity with quality.
???????????????
Brochure
?DevOps defined
The challenge
Driving collaboration HPE approach
Four keys
HPE Software Professional Services
Driving collaboration across the development lifecycle and IT operations
DevOps is all about people, process, and technology, and applying principles and methods to drive better collaboration between your software delivery and IT operations teams. DevOps extends the agile mindset
to incorporate operations. Whether your organization uses agile, waterfall, or both, DevOps recognizes the interdependences of application delivery and IT operations in accelerating the release of high-quality software.
DevOps enables continuous delivery, which focuses on what is ultimately most important: shorter cycles for putting high-quality software into the hands of end users. Continuous delivery relies on better collaboration, integrated processes, and the comprehensive automation of build, test, and deployment processes to make software ready for release on demand.
Key benefits of DevOps
Here are some of the ways that DevOps helps you drive business value.
• Gain a competitive advantage. Accelerate the release of applications into production. Respond faster to business demand.
• Increase IT resource efficiency. Automate provisioning and deployment. Remove manual processes. • Enable better and faster decisions. Create an immediate feedback loop. Identify issues earlier in
the process.
• Keep pace with business demands. Bring new apps and updates to market quickly to create satisfied customers.
???????????????
Brochure
?DevOps defined
The challenge
Driving collaboration HPE approach
Four keys
HPE Software Professional Services
A holistic approach to DevOps
Many technology vendors and service providers can deliver pieces and parts of DevOps, but not the full solution. That’s not the case with Hewlett Packard Enterprise. Drawing on a vast software portfolio and the broad reach of the HPE Software Professional Services organization, HPE delivers end-to-end solutions for DevOps.
The holistic HPE Software Professional Services approach to DevOps spans from planning and definition through monitoring and optimization of your in-production applications. It extends agile principles to:
• Eliminate silos across your organization
• Enable cross-team collaboration
• Automate build, test, and deployment processes
• Accelerate release cycles
• Test and monitor earlier in the development process and test in production
????????????????
Brochure
?DevOps defined
The challenge
Driving collaboration HPE approach
Four keys
HPE Software Professional Services
Get on the path to DevOps with these four keys
DevOps brings fundamental changes to the way application delivery and operations teams interact and carry out processes. It requires changes to technology, processes, and culture, and change at this level can be challenging if not addressed in a systematic manner.
So how do you approach DevOps? HPE Software Professional Services recommends that you begin your DevOps journey by focusing on four keys to a successful DevOps implementation.
1. Assess your DevOps strategy.
2. Identify the DevOps maturity of your core development and IT operations processes. 3. Adopt and implement change to improve velocity.
4. Measure progress and plan next improvement.
???????????????
Brochure
Key 1. Assess your DevOps strategy
The first step on the path to DevOps is an assessment of your DevOps strategy and what you are trying to achieve. Gartner adds these thoughts on the assessment process: “Before beginning a DevOps initiative, an assessment of current systemic issues (and technical debt) is needed to develop a common understanding of costs and intended efficiency gains to assess what needs fixing.”1
In more specific terms, the assessment process helps you:
• Identify your DevOps business drivers, challenges, and initiatives
• Examine critical success factors and best practices from successful projects • Review critical DevOps domains that align to your business goals
• Identify priorities for short, midsize, and long-term projects
Read the HPE DevOps Transformation Workshop Service brief.
??“What are we trying to achieve?”
1 Gartner, “Seven Steps to Start Your DevOps Initiative,” September 16, 2014.
Brochure
?“How far along are we?”
Key 2. Identify the DevOps maturity of your core development and IT operations processes
To put your organization on the path to DevOps, you need a clear understanding of your current practices for development and operations. Do your development processes use agile or waterfall methodologies, or both? Do your operations teams follow ITIL processes? What are your current practices for planning and communication?
In some specific areas of focus, the exploration of development practices looks at your approaches to application decoupling, continuous integration and the use of an apps hub, test automation, good coding practices, configurability, and maintainability. The review of operations practices explores your approaches to an operations hub, infrastructure as code, automated release management, resolution practices, and monitoring.
Ultimately, this process helps you:
• Identify key stakeholders in business, development, QA, and operations
• Determine the app development methodologies—agile, waterfall, both—that are most important • Determine the ITIL processes that are most important
• Understand your planning and communication practices
Read the HPE DevOps Solution Discovery Workshop Service brief.
?
Brochure
Managed
DevOps maturity model: a proven approach
To understand the DevOps maturity of your core development and IT operations processes, HPE Software Professional Services leverages a proven DevOps maturity model. This model looks at DevOps from three viewpoints—process, automation, and collaboration—and spans a series of clearly defined states on the path to an optimized DevOps environment.
The DevOps maturity exploration leaves you with an understanding of your organization’s maturity level in terms of process standardization, automation tools, and collaboration approaches, along with insights into your opportunities for improvement.
?Defined
Optimized
??Measured
???????Initial
??Collaboration-based processes are measured to identify inefficiencies and bottlenecks
Effective knowledge sharing and individual empowerment
?Collaboration, shared decision making and accountability
????Managed
communication, some shared decision making
Collect and analyze metrics of the automated processes and measure against the business goals
Self-service automation, self-learning using analytics and self-remediation
??Poor, ad hoc communication and coordination
Central automated processes across the application lifecycle
?Siloed automation, no central infrastructure
????No automation
?Process risk and cost optimization
?Processes are standardized across the organization
Visibility and predictability of entire process quality and performance
?Unpredictable, uncontrolled reactive processes
Processes are managed but not standardized
Brochure
Key 3. Adopt and implement change to improve velocity
DevOps encompasses the entire application lifecycle. The ultimate goal is to ensure that apps can be deployed into production in an agile, automated, and continuous manner.
To that end, the DevOps journey determines areas of standards and automation and then implements them in a phased approach. The ultimate goal is to move your organization to accelerate at the speed of business. Here we focus on the people, process and technology around four key DevOps areas: (1) continuous assessment, (2) continuous integration and testing, (3) continuous delivery and deployment and (4) continuous operations. Organizational change and cultural change is addressed throughout.
Read the HPE DevOps Services brief.
??“Where can we change?”
Brochure
??“What will our results look like?”
Key 4. Measure progress and plan next improvement
Another key to the journey to DevOps is to identify the measures and metrics you will use to gauge the success of your initiative, such as business enablement, cycle times, and escaped defects. In many cases,
your DevOps metrics will be different from those used today in app delivery and IT operations. The goal is to establish a common set of business-driven metrics that incorporates development, QA, and operational views.
Gartner offers this guidance: “Build new shared metrics that align with the business needs and impacts, but, more importantly, help people realize that they must work together. Goals should be service-focused, with an eye toward improving agility (velocity) and improving business value (quality).”2
“In a DevOps initiative, the metrics must be aligned to the end goal—business enablement via faster production release.”
—Gartner2
?2 Gartner, “Seven Steps to Start Your DevOps Initiative,” September 16, 2014.
Brochure
?DevOps defined
The challenge
Driving collaboration HPE approach
Four keys
HPE Software Professional Services
Make your journey to DevOps with HPE Software Professional Services
With its holistic approach to DevOps, extensive software services portfolio, and unique intellectual property, HPE Software Professional Services is your ideal partner for your journey to DevOps. HPE offers all the resources you need to bring together your people, processes, and technology to enable unified application delivery and increase business agility.
Better still, the HPE approach to DevOps is designed to help you to gain business value at each incremental step in your journey.
?????????People
HPE DevOps
Services
• End-to-end implementation
• Strategy and vision workshops • Maturity assessments
• Reference architecture
?????????• •
Best practices:
process and
methodologies Proces
Process automation
and workflows
Technology
• HPE product deployment and third-party integrations
• Templates and accelerators
?????????s
???
Brochure
Let’s get started
In today’s application-driven world, your competitive edge increasingly relies on your ability to continually deliver high-quality software to end users. That’s a key value enabled by DevOps. Through its ability to bring together application delivery and IT operations, DevOps gives your business the agility it needs to compete successfully in the era of the app.
Ready for a deeper dive?
For more information on HPE Software Professional Services offerings for your DevOps journey, visit us at hp.com/go/agilityservices.
??????Sign up for updates
Rate this document
???© Copyright 2014–2015 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for HPE products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HPE shall not be liable for technical or editorial errors or omissions contained herein.
4AA5-5748ENW, November 2015, Rev. 1
??