Technology

Triple E class DevOps with Hudson, Maven, Kokki/Multiconf and PyDev

Description
At Maersk Line, not only the world's biggest container ships, the 'Triple-E' class vessels were built. Continuous Integration and Delivery on a similar scale using Hudson, Maven and tools like Kokki (similar to Puppet or Chef, but written in Python) are also practiced there. This session is going to give a brief overview of Multi-Configuration (Matrix) job types used in most projects at Maersk around the globe. Things are being built and deployed in a heterogenous environment, otherwise probably found only at very large vendors of Public Cloud services like Google or Amazon. Provisioning of various OS is automated through Vagrant. Management and Planning of all tasks and 'Sprints' is following Agile principles, especially DevOps style Kanban. Where possible planned and controlled by Eclipse-based tools such as Mylyn Connectors accessing planning tools like TeamConcert, Xplanner or Mantis. While feature projects use Eclipse for Java or Scala/Play!, the DevOps teams use PyDev for Jython/WSTL or Python development.
Categories
Published
of 35
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  • 1. Triple-E’ class DevOps with Hudson, Maven, Kokki Multiconf & PyDev Werner Keil PyUGAT 15th October 2014
  • 2. Images © 2 © 2012-2013 Creative Arts & Technologies Maersk Line and Others
  • 3. Overview • Introduction • Kokki • Configuration • Multiconf • PyDev • Development • Maven • Build • Hudson • Automation • Demo •Q&A 3 © 2012-2013 Creative Arts & Technologies
  • 4. Who am I? Werner Keil • Consultant – Coach • Creative Cosmopolitan • Open Source Evangelist • Software Architect • Java Godfather • UOMo Project Lead • Mærsk DevOps Guy … Email werner@catmedia.us 4 © 2012-2013 Creative Arts & Technologies Twitter @wernerkeil
  • 5. Kokki • Infrastructure Automation → Configuration Management • Part of Provisioning • Library and simple command line tool • Currently no client/server component • Can use GitHub in place of client/server • Alternative to Puppet/Chef/Cfengine • Implemented in Python 5 © 2012 Creative Arts & Technologies
  • 6. Kokki | Terms • Kokki is a configuration management framework. • It can be used standalone or as a part of a more complex system. • It means “cook” in Finnish • Environment: Execution environment • Resource: Describes a file, service, package, etc.. • Provider: Knows how to execute a resource • Kitchen: Container for cookbooks • Cookbook: Container for recipes and libraries • Recipe: group of resource definitions • Library: utility methods, resources, and providers 6 © 2012 Creative Arts & Technologies
  • 7. Kokki | Quick Example from kokki import * with Environment() as env: Package("vim", action="upgrade") File("/etc/hosts", owner = "root", group = "root", content = "127.0.0.1 localhostn" "255.255.255.255 broadcasthostn" "::1 localhostn" "fe80::1%lo0 localhostn") env.run() 7 © 2012 Creative Arts & Technologies
  • 8. Kokki++ | Multiconf •Multiconf is a framework for describing complex environments using Python. •Why • It started from a simple need of continuous deployment of Java EE projects. • Using a bunch of Python scripts, and a bunch of plain text property files. • When number of property files increased number of scripts increased, too… • Out of this dilemma Multiconf was born. 8 © 2012 Creative Arts & Technologies
  • 9. Multiconf •How • Imagine a project, using multiple servers like Tomcat, Jetty, JBoss, Glassfish or Weblogic. • You are going to use database connections and four environments: • Local (for developer's local machine) • Dev • Test • Prod • Project configuration will be similar on different environments, but something will be different (databases and ports, for example). 9 © 2012 Creative Arts & Technologies
  • 10. Multiconf (2) •What Multiconf is not • Multiconf itself doesn't know how to create environments. • Multiconf doesn't know how to create any of the environment's components • Multiconf has nothing to execute • Perfect match for a system such as Kokki • Same language • Pluggable recipes and Libraries make Kokki open to such Extensions •What Multiconf is • Multiconf allows you to define your software stack and get early warning that something within your definition is wrong. Other tools use YAML or JSON to define settings of the components, but then you need something to validate those settings. Multiconf is both - definition and validation. 10 © 2012 Creative Arts & Technologies
  • 11. Multiconf | JSF Example from javax_faces import ProjectStage from valid_envs import local, dev, test, pp, prod def jsf_stages(): with ProjectStage() as ps: ps.name(local=”Development”, dev=”Development”, test=”UnitTest”, pp=”SystemTest”, prod=”Production”) 11 © 2012 Creative Arts & Technologies
  • 12. Multiconf | Inspired... Apache Tamaya • Started by • Anatole Tresch (Crédit Suisse) • Mit etwas Hilfe u.a. durch den „Java Godfather“ •Proposal for • An Apache Incubator Project • See.: https://github.com/java-config © 2012-2014 Creative Arts & Technologies
  • 13. Multiconf | ”at Apache” Apache Tamaya •Scope • Provide type-safe enterprise-ready configuration standard for SE and EE (optionally also ME?) • Ensure it is a simple but extensible • Ready for complex scenarios like Java EE, Multi-Tenancy etc. • Ensure interoperability with existing solutions • Align Configuration Aspects with EE8 and the Java Platform • Other languages (as Apache is rather polyglot) © 2012-2014 Creative Arts & Technologies
  • 14. Multiconf | ”at Apache” Apache Tamaya • State • Already running code base • Large support in User Groups around the world • Supported by leading industry experts © 2012-2014 Creative Arts & Technologies
  • 15. PyDev •PyDev is a product of Appcelerator, a platform and services company that is enabling Web developers to build cross-platform / native applications for Mobile, Tablet and Desktop platforms Winner of 2011 Eclipse Award for Best Developer Tool Appcelerator also makes Aptana Studio and Titanium Studio, Eclipse-based IDEs targeting Web and Mobile platforms. 15 © 2012 Creative Arts & Technologies
  • 16. PyDev Now •PyDev ist now Open Source Project, lead by Fabio Zadrozny in Florianópolis, Brasil: www.pydev.org © 2012-2014 Creative Arts & Technologies Bilder © FIFA
  • 17. PyDev | Code Completion 17 © 2012 Creative Arts & Technologies
  • 18. PyDev | Code Navigation 18 © 2012 Creative Arts & Technologies
  • 19. PyDev | Code Coverage 19 © 2012 Creative Arts & Technologies
  • 20. PyDev | Debugging 20 © 2012 Creative Arts & Technologies
  • 21. PyDev | Unit Tests 21 © 2012 Creative Arts & Technologies
  • 22. PyDev DEMO
  • 23. Maven The evolution of build systems 23 © 2012 Creative Arts & Technologies
  • 24. Maven “Lean” Usage • Using Maven for what it’s Best at • Dependency Management • Nexus •Managed Environment • Generated / Called by Kokki • Hudson Plugins 24 © 2012 Creative Arts & Technologies
  • 25. Hudson •Hudson is a powerful and widely used open source continuous integration server providing development teams with a reliable way to monitor changes in source control and trigger a variety of builds. (The Hudson Book) 25 © 2012 Creative Arts & Technologies
  • 26. Hudson | Downloads 26 © 2012 Creative Arts & Technologies
  • 27. Hudson | Commits 27 © 2012 Creative Arts & Technologies
  • 28. Hudson | Job Coordination 28 © 2012 Creative Arts & Technologies
  • 29. Hudson | Job Coordination Triple-E size 29 © 2012 Creative Arts & Technologies
  • 30. Hudson | Promoted Builds 30 © 2012 Creative Arts & Technologies
  • 31. The Price of Continuous Integration 32 © 2012 Creative Arts & Technologies
  • 32. Let‘s talk Q & A
  • 33. Links Kokki https://github.com/samuel/kokki Multiconf https://github.com/lhupfeldt/multiconf PyDev http://www.pydev.org
  • 34. Links (2) Eclipse – Hudson http://www.eclipse.org/hudson/ Apache Maven http://www.apache.org/maven/
  • 35. Twitter @wernerkeil Thank you Email werner@catmedia.us 36 © 2012 Creative Arts & Technologies Images © Maersk Line
  • Search
    Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks