Showing posts with label Packt. Show all posts

Over 70 simple but incredibly effective recipes for taking control of automated testing using powerful Python testing tools

Greg L. Turnquist

e-books shop
e-books shop
Purchase Now !
Just with Paypal

Book Details
 362 p
 File Size 
 8,659 KB
 File Type
 PDF format
 2011 Packt Publishing 

About the Author
Greg L. Turnquist has worked in the software industry since 1997. He is an active
participant in the open source community, and has contributed patches to several projects
including MythTV, Spring Security, MediaWiki, and the TestNG Eclipse plugin. As a test-bitten
script junky, he has always sought the right tool for the job. He is a firm believer in agile
practices and automated testing. He has developed distributed systems, LAMP-based setups,
and supported mission-critical systems hosted on various platforms.
After graduating from Auburn University with a Master's in Computer Engineering, Greg
started working with Harris Corporation. He worked on many contracts utilizing many types
of technology. In 2006, he created the Spring Python project and went on to write Spring
Python 1.1 in 2010. He joined SpringSource, a division of VMware in 2010, as part of their
international software development team.

About the Reviewers
Matthew Closson is a creative technologist and entrepreneur at heart. He is currently
employed as a software engineer by Philips Healthcare. He is passionate about software
testing, systems integration, and web technologies. When not obsessing over Ruby and C#
code, this elusive developer is likely to be found reading at the local bookstore or relaxing on the beach.

Chetan Giridhar has more than five years experience of working in the software services
industry, product companies, and research organizations. He has a string background of
C/C++, Java (certified Java professional) and has a good command of Perl, Python scripting
languages, using which he has developed useful tools and automation frameworks. His
articles on Code Reviews, Software Automation, and Agile methodologies have been
published in international magazines including TestingExperience and AgileRecord for which
he has received appreciation from other industry experts on his website—TechnoBeans.
Chetan has also co-authored a book on Design Patterns in Python that is listed at Python's
Official Website. He has given lectures on Python Programming to software professionals
and at educational institutes including the Indian Institute of Astrophysics, Bangalore. Chetan
holds a B.E. in Electrical Engineering from the University of Mumbai and feels that the world is
full of knowledge.

Sylvain Hellegouarch is a senior software engineer with several years experience in
development and performance testing in various companies, both in France and in the United
Kingdom. Passionate about open-source software, he has written several Python projects
around communication protocols such as HTTP, XMPP, and the Atom Publishing Protocol. He
has been part of the CherryPy team since 2004 and has also authored the CherryPy Essentials
book, published by Packt Publishing in 2007. Sylvain also reviewed Spring Python, published
by Packt Publishing in 2010. His current interests are set on the open-data movement and the
wave of innovation it brings to public services. When away from his computer, Sylvain plays the
guitar and the drums or spends his time with friends and family.

Maurice HT Ling completed his Ph.D. in Bioinformatics and B.Sc(Hons) in Molecular and
Cell Biology from The University of Melbourne where he worked on microarray analysis and
text mining for protein-protein interactions. He is currently a Senior Scientist (Bioinformatics)
in Life Technologies and an Honorary Fellow in The University of Melbourne, Australia.
Maurice holds several Chief Editorships including The Python Papers, Computational and
Mathematical Biology, and Methods and Cases in Computational, Mathematical, and
Statistical Biology. In Singapore, he co-founded the Python User Group (Singapore) and has
been the co-chair of PyCon Asia-Pacific since 2010. In his free time, Maurice likes to train in
the gym, read, and enjoy a good cup of coffee. He is also a Senior Fellow of the International
Fitness Association, USA. His personal website is:

Testing has always been a part of software development. For decades, comprehensive testing
was defined by complex manual test procedures backed by big budgets; but something
revolutionary happened in 1998. In his Guide to Better Smalltalk, Smalltalk guru Kent Beck
introduced an automated test framework called SUnit. This triggered an avalanche of test
frameworks including JUnit, PyUnit, and many others for different languages and various
platforms, dubbed the xUnit movement. Automated testing was made a cornerstone
of the agile movement when 17 top software experts signed the Agile Manifesto in 2001.
Testing includes many different styles including unit testing, integration testing, acceptance
testing, smoke testing, load testing, and countless others. This book digs in and explores
testing at all the important levels while using the nimble power of Python. It also shows many tools.

This book is meant to expand your knowledge of testing from something you either heard
about or have practiced a little into something you can apply at any level to meet your needs
in improving software quality. I hope to give you the tools to reap huge rewards in better
software development and customer satisfaction.

Table of Contents
Preface 1
Chapter 1: Using Unittest To Develop Basic Tests 5
Introduction 5
Asserting the basics 7
Setting up and tearing down a test harness 11
Running test cases from the command line with increased verbosity 14
Running a subset of test case methods 16
Chaining together a suite of tests 18
Defining test suites inside the test module 21
Retooling old test code to run inside unittest 25
Breaking down obscure tests into simple ones 29
Testing the edges 35
Testing corner cases by iteration 39
Chapter 2: Running Automated Test Suites with Nose 45
Introduction 45
Getting nosy with testing 46
Embedding nose inside Python 49
Writing a nose extension to pick tests based on regular expressions 52
Writing a nose extension to generate a CSV report 59
Writing a project-level script that lets you run different test suites 66
Chapter 3: Creating Testable Documentation with doctest 77
Introduction 77
Documenting the basics 78
Catching stack traces 82
Running doctests from the command line 85
Coding a test harness for doctest 88
Filtering out test noise 92
Printing out all your documentation including a status report 96
Testing the edges 101
Testing corner cases by iteration 104
Getting nosy with doctest 107
Updating the project-level script to run this chapter's doctests 110
Chapter 4: Testing Customer Stories with Behavior
Driven Development 117
Introduction 117
Naming tests that sound like sentences and stories 120
Testing separate doctest documents 126
Writing a testable story with doctest 130
Writing a testable novel with doctest 136
Writing a testable story with Voidspace 142
Mock and nose 142
Writing a testable story with mockito and nose 147
Writing a testable story with Lettuce 150
Using Should DSL to write succinct assertions with Lettuce 158
Updating the project-level script to run this chapter's BDD tests 163
Chapter 5: High Level Customer Scenarios with Acceptance Testing 169
Introduction 170
Installing Pyccuracy 172
Testing the basics with Pyccuracy 176
Using Pyccuracy to verify web app security 179
Installing the Robot Framework 183
Creating a data-driven test suite with Robot 186
Writing a testable story with Robot 191
Tagging Robot tests and running a subset 197
Testing web basics with Robot 204
Using Robot to verify web app security 208
Creating a project-level script to verify this chapter's acceptance tests 212
Chapter 6: Integrating Automated Tests with Continuous Integration 217
Introduction 217
Generating a continuous integration report for Jenkins using NoseXUnit 220
Configuring Jenkins to run Python tests upon commit 222
Configuring Jenkins to run Python tests when scheduled 227
Generating a CI report for TeamCity using teamcity-nose 231
Configuring TeamCity to run Python tests upon commit 234
Configuring TeamCity to run Python tests when scheduled 237
Chapter 7: Measuring your Success with Test Coverage 241
Introduction 241
Building a network management application 243
Installing and running coverage on your test suite 251
Generating an HTML report using coverage 255
Generating an XML report using coverage 257
Getting nosy with coverage 259
Filtering out test noise from coverage 261
Letting Jenkins get nosy with coverage 264
Updating the project-level script to provide coverage reports 269
Chapter 8: Smoke/Load Testing—Testing Major Parts 275
Introduction 275
Defining a subset of test cases using import statements 277
Leaving out integration tests 281
Targeting end-to-end scenarios 285
Targeting the test server 290
Coding a data simulator 298
Recording and playing back live data in real time 303
Recording and playing back live data as fast as possible 311
Automating your management demo 319
Chapter 9: Good Test Habits for New and Legacy Systems 323
Introduction 324
Something is better than nothing 324
Coverage isn't everything 326
Be willing to invest in test fixtures 328
If you aren't convinced on the value of testing, your team
won't be either 330
Harvesting metrics 331
Capturing a bug in an automated test 332
Separating algorithms from concurrency 333
Pause to refactor when test suite takes too long to run 334
Cash in on your confidence 336
Be willing to throw away an entire day of changes 337
Instead of shooting for 100 percent coverage, try to have a steady growth 339
Randomly breaking your app can lead to better code 340
Index 343

e-books shop

What you need for this book
You will need Python 2.6 or above. The recipes in this book have NOT been tested against
Python 3+. This book uses many other Python test tools, but includes detailed steps to show
how to install and use them.
Who this book is for
This book is for Python developers who want to take testing to the next level. It covers different
styles of testing, giving any developer an expanded set of testing skills to help write better
systems. It also captures lessons learned from the author, explaining not only how to write
better tests but why.

Over 80 object-oriented recipes to help you create mind-blowing GUIs in Python

Burkhard A. Meier

e-books shop
e-books shop
Purchase Now !
Just with Paypal

Book Details
 351 p
 File Size 
 9,239 KB
 File Type
 PDF format
 2015 Packt Publishing 

About the Author
Burkhard A. Meier has more than 15 years of professional experience working in the
software industry as a software tester and developer, specializing in software test automation
development, execution, and analysis. He has a very strong background in SQL relational
database administration, the development of stored procedures, and debugging code.
While experienced in Visual Studio .NET C#, Visual Test, TestComplete, and other testing
languages (such as C/C++), the main focus of the author over the past two years has been
developing test automation written in Python 3 to test the leading edge of FLIR ONE infrared
cameras for iPhone and Android smart phones as well as handheld tablets.

Being highly appreciative of art, beauty, and programming, the author developed GUIs in C# and
Python to streamline everyday test automation tasks, enabling these automated tests to run
unattended for weeks, collecting very useful data to be analyzed and automatically plotted into
graphs and e-mailed to upper management upon completion of nightly automated test runs.
His previous jobs include working as a senior test automation engineer and designer for
InfoGenesis (now Agilysys), QAD, InTouch Health, and presently, FLIR Systems.
You can get in touch with him through his LinkedIn account, 

About the Reviewers
Joy Bindroo holds a bachelor's degree in computer science and engineering. He is currently
pursuing his post-graduate studies in the field of information management. He is a creative
person and enjoys working on Linux platform and other open source technologies. He
enjoys writing about Python and sharing his ideas and skills on his website,
He likes to sketch, write poems, listen to music, and have fun with his
friends in his free time.

Peter Bouda works as a senior web developer for MAJ Digital and is a specialist in
full stack JavaScript applications based on LoopBack and AngularJS. He develops Python
GUIs for companies and research projects since 2003 and wrote a German book, PyQt und
PySide – GUI- und Anwendungsentwicklung mit Python und Qt, on Python GUI development,
which was published in 2012. Currently, he is getting crazy with embedded and open
hardware platforms and is working on a modular game console based on open hardware.

Joseph Rex is a full stack developer with a background in computer security. He has worked
on Python GUI products and some CLI programs to experiment with information security. He
came out of security to web development and developed a passion for rails and JavaScript MVC
frameworks after working on several projects using jQuery. He has been in the web industry
for 3 years, building web applications and mobile apps. He has also written articles on security
for InfoSec Institute and has written some scripts to back them up. He has to his credit several
personal experimental projects written in Python.

In this book, we will explore the beautiful world of graphical user interfaces (GUIs) 
using the Python programming language.
Along the way, we will talk to networks, queues,
 the OpenGL graphical library, and many more technologies.
This is a programming cookbook. Every chapter is self-contained 
and explains a certain programming solution.
We will start very simply, yet throughout this book 
we will build a working program written in Python 3.
We will also apply some design patterns and use best practices throughout this book.
The book assumes that the reader has some basic experience using the Python
programming language, but that is not really required to use this book.
If you are an experienced programmer in any programming language, you will have a fun

time extending your skills to programming GUIs using Python!

Are you ready?

Let's start on our journey…

What you need for this book
All required software for this book is available online and is free of charge. This starts with
Python 3 itself, and then extends to Python's add-on modules. In order to download any
required software, you will need a working Internet connection.

Table of Contents
Preface v
Chapter 1: Creating the GUI Form and Adding Widgets 1
Introduction 1
Creating our first Python GUI 2
Preventing the GUI from being resized 4
Adding a label to the GUI form 6
Creating buttons and changing their text property 7
Text box widgets 9
Setting the focus to a widget and disabling widgets 11
Combo box widgets 12
Creating a check button with different initial states 14
Using radio button widgets 16
Using scrolled text widgets 18
Adding several widgets in a loop 20
Chapter 2: Layout Management 23
Introduction 23
Arranging several labels within a label frame widget 24
Using padding to add space around widgets 26
How widgets dynamically expand the GUI 28
Aligning the GUI widgets by embedding frames within frames 32
Creating menu bars 36
Creating tabbed widgets 41
Using the grid layout manager 46
Chapter 3: Look and Feel Customization 49
Introduction 49
Creating message boxes – information, warning, and error 50
How to create independent message boxes 53
How to create the title of a tkinter window form 56
Changing the icon of the main root window 57
Using a spin box control 58
Relief, sunken, and raised appearance of widgets 61
Creating tooltips using Python 63
How to use the canvas widget 67
Chapter 4: Data and Classes 69
Introduction 69
How to use StringVar() 69
How to get data from a widget 73
Using module-level global variables 75
How coding in classes can improve the GUI 79
Writing callback functions 85
Creating reusable GUI components 86
Chapter 5: Matplotlib Charts 91
Introduction 91
Creating beautiful charts using Matplotlib 92
Matplotlib – downloading modules using pip 94
Matplotlib – downloading modules with whl extensions 98
Creating our first chart 100
Placing labels on charts 102
How to give the chart a legend 106
Scaling charts 109
Adjusting the scale of charts dynamically 112
Chapter 6: Threads and Networking 117
Introduction 117
How to create multiple threads 118
Starting a thread 121
Stopping a thread 125
How to use queues 128
Passing queues among different modules 133
Using dialog widgets to copy files to your network 136
Using TCP/IP to communicate via networks 145
Using URLOpen to read data from websites 147
Chapter 7: Storing Data in Our MySQL Database via Our GUI 153
Introduction 153
Connecting to a MySQL database from Python 154
Configuring the MySQL connection 157
Designing the Python GUI database 161
Using the SQL INSERT command 168
Using the SQL UPDATE command 172
Using the SQL DELETE command 177
Storing and retrieving data from our MySQL database 181
Chapter 8: Internationalization and Testing 187
Introduction 187
Displaying widget text in different languages 188
Changing the entire GUI language all at once 191
Localizing the GUI 196
Preparing the GUI for internationalization 201
How to design a GUI in an agile fashion 204
Do we need to test the GUI code? 208
Setting debug watches 212
Configuring different debug output levels 216
Creating self-testing code using
Python's __main__ section 220
Creating robust GUIs using unit tests 224
How to write unit tests using the Eclipse PyDev IDE 229
Chapter 9: Extending Our GUI with the wxPython Library 235
Introduction 235
How to install the wxPython library 236
How to create our GUI in wxPython 239
Quickly adding controls using wxPython 244
Trying to embed a main wxPython app in a main tkinter app 251
Trying to embed our tkinter GUI code into wxPython 253
How to use Python to control two different GUI frameworks 256
How to communicate between the two connected GUIs 260
Chapter 10: Creating Amazing 3D GUIs with PyOpenGL and PyGLet 265
Introduction 265
PyOpenGL transforms our GUI 266
Our GUI in 3D! 270
Using bitmaps to make our GUI pretty 275
PyGLet transforms our GUI more easily than PyOpenGL 279
Our GUI in amazing colors 283
Creating a slideshow using tkinter 286
Chapter 11: Best Practices 291
Introduction 291
Avoiding spaghetti code 291
Using __init__ to connect modules 298
Mixing fall-down and OOP coding 305
Using a code naming convention 310
When not to use OOP 314
How to use design patterns successfully 317
Avoiding complexity 320
Index 327

e-books shop

Who this book is forI
This book is for programmers who wish to create a graphical user interface (GUI). You might
be surprised by what we can achieve by creating beautiful, functional, and powerful GUIs
using the Python programming language. Python is a wonderful, intuitive programming
language, and is very easy to learn.

I like to invite you to start on this journey now. It will be a lot of fun!

Unleash the power of Python 3 objects

Dusty Phillips

e-books shop
e-books shop
Purchase Now !
Just with Paypal

Book Details
 460 p
 File Size 
 2,619 KB
 File Type
 PDF format
 2015 Packt Publishing 

About the Author
Dusty Phillips is a Canadian software developer and author currently living in
Seattle, Washington. He has been active in the open source community for a decade
and a half and programming in Python for nearly all of it. He cofounded the popular
Puget Sound Programming Python meetup group; drop by and say hi if you're in the area.
Python 3 Object Oriented Programming, Packt Publishing, was the first of his books. He
has also written Creating Apps In Kivy, O'Reilly, the mobile Python library, and selfpublished
Hacking Happy, a journey to mental wellness for the technically inclined.
He was hospitalized for suicidal tendencies shortly after the first edition of this book
was published and has been an outspoken proponent for positive mental health
ever since.

About the Reviewers
AMahdy AbdElAziz has more than 8 years of experience in software engineering
using several languages and frameworks. Over the last 5 years, he has focused
on Android and mobile development, including cross-platform tools, and
Android internals, such as building custom ROMs and customizing AOSP
for embedded devices.
He is currently teaching Python at Information Technology Institution. You can
visit his website,, to find out more about him.

Grigoriy Beziuk is a former CIO of Crowdage Foundation, acting as an
independent software developer as this book was being written. He has worked
with a wide variety of programming languages and technologies, including
different versions of Python in different environments, ranging from purely
scientific ones to modern production-scale web development issues.

Krishna Bharadwaj is the cofounder of SMERGERS (, 
a Fintech start-up helping small and medium businesses raise capital from
investors and different financial institutions. In the past, he has worked with early
stage start-ups such as BlockBeacon (Santa Monica) and PricePoint (CA) and large
organizations such as National Instruments, Bangalore, and Google, New York.
Krishna got introduced to Python and FOSS during his college days and has continued
to use it extensively in his personal projects and also professionally at work. Because
of his liking for teaching and mentoring, he visits different universities, conducting
workshops whenever he gets an opportunity.
He holds a master's degree in computer science from the University of Southern
California, Los Angeles, and a bachelor's degree in information science and
engineering from the BMS College of Engineering, Bangalore. He can be reached
through his e-mail,, or his website,

Justin Cano is a recent graduate from the University of California, Riverside,
with a BS in computer engineering and is currently working as a software engineer
in the Silicon Valley area with hopes of moving to a big tech company such as
Google or Apple.
He first started programming in the sixth grade, creating small, primitive websites
in HTML and CSS. He started to learn computer science theory and C++ in his first
year at UC Riverside and then started learning Python in his third year.
Justin admits that at first, he wasn't immediately attracted to Python, since abstractions
between C++ and Python are very different. It wasn't until he began to express more of
an interest in coding contests and challenges that he began to show interest in Python,
mainly because he feels that the readability and elegance of the Python syntax allows
him to quickly and more naturally turn ideas and thought processes into Python
code. He now writes Python code regularly, often to create mock-ups or prototypes of
software applications before moving on to a more domain-specific language.

Anthony Petitbois is an online architect in the video game industry with 13 years
of professional experience in operations and development and more than 20 years
of software development experience. He is passionate about new technologies and
loves to take creative approaches to solve complex problems.
In his spare time, he learns new languages and new platforms, plays video games,
and spends time with his family in the beautiful region of British Columbia, Canada,
where he now lives after emigrating from France in 2009.

Claudio Rodriguez started working on PLCs for GE, but his main goal has always
been research and development and turning dreams into reality. This made him
move from automation engineering to software engineering and the structured way
of software, OOD; the remote team working from the comfort of his computer was
just too powerful not to take advantage of. During his master's, he got to learn the
proper place to look for resources and found a friend in books and research papers
and conferences. Eventually, he started working on a system to control an electric
arc furnace, but the needs of his clients moved him into taking further control of
technology. He has a deep love for complex AI and can be seen surrounded by
papers, books, and a computer to test things, but he keeps things real by delivering
beautiful and dynamic applications for his customers.

Table of Contents
Preface vii
Chapter 1: Object-oriented Design 1
Introducing object-oriented 1
Objects and classes 3
Specifying attributes and behaviors 5
Data describes objects 6
Behaviors are actions 7
Hiding details and creating the public interface 9
Composition 11
Inheritance 14
Inheritance provides abstraction 16
Multiple inheritance 17
Case study 18
Exercises 25
Summary 26
Chapter 2: Objects in Python 27
Creating Python classes 27
Adding attributes 29
Making it do something 30
Talking to yourself 30
More arguments 31
Initializing the object 33
Explaining yourself 35
Modules and packages 37
Organizing the modules 40
Absolute imports 40
Relative imports 41
Organizing module contents 43
Who can access my data? 46
Third-party libraries 48
Case study 49
Exercises 58
Summary 58
Chapter 3: When Objects Are Alike 59
Basic inheritance 59
Extending built-ins 62
Overriding and super 63
Multiple inheritance 65
The diamond problem 67
Different sets of arguments 72
Polymorphism 75
Abstract base classes 78
Using an abstract base class 78
Creating an abstract base class 79
Demystifying the magic 81
Case study 82
Exercises 95
Summary 96
Chapter 4: Expecting the Unexpected 97
Raising exceptions 98
Raising an exception 99
The effects of an exception 101
Handling exceptions 102
The exception hierarchy 108
Defining our own exceptions 109
Case study 114
Exercises 123
Summary 124
Chapter 5: When to Use Object-oriented Programming 125
Treat objects as objects 125
Adding behavior to class data with properties 129
Properties in detail 132
Decorators – another way to create properties 134
Deciding when to use properties 136
Manager objects 138
Removing duplicate code 140
In practice 142
Case study 145
Exercises 153
Summary 154
Chapter 6: Python Data Structures 155
Empty objects 155
Tuples and named tuples 157
Named tuples 159
Dictionaries 160
Dictionary use cases 164
Using defaultdict 164
Counter 166
Lists 167
Sorting lists 169
Sets 173
Extending built-ins 177
Queues 182
FIFO queues 183
LIFO queues 185
Priority queues 186
Case study 188
Exercises 194
Summary 195
Chapter 7: Python Object-oriented Shortcuts 197
Python built-in functions 197
The len() function 198
Reversed 198
Enumerate 200
File I/O 201
Placing it in context 203
An alternative to method overloading 205
Default arguments 206
Variable argument lists 208
Unpacking arguments 212
Functions are objects too 213
Using functions as attributes 217
Callable objects 218
Case study 219
Exercises 226
Summary 227
Chapter 8: Strings and Serialization 229
Strings 229
String manipulation 230
String formatting 232
Escaping braces 233
Keyword arguments 234
Container lookups 235
Object lookups 236
Making it look right 237
Strings are Unicode 239
Converting bytes to text 240
Converting text to bytes 241
Mutable byte strings 243
Regular expressions 244
Matching patterns 245
Matching a selection of characters 246
Escaping characters 247
Matching multiple characters 248
Grouping patterns together 249
Getting information from regular expressions 250
Making repeated regular expressions efficient 252
Serializing objects 252
Customizing pickles 254
Serializing web objects 257
Case study 260
Exercises 265
Summary 267
Chapter 9: The Iterator Pattern 269
Design patterns in brief 269
Iterators 270
The iterator protocol 271
Comprehensions 273
List comprehensions 273
Set and dictionary comprehensions 276
Generator expressions 277
Generators 279
Yield items from another iterable 281
Coroutines 284
Back to log parsing 287
Closing coroutines and throwing exceptions 289
The relationship between coroutines, generators, and functions 290
Case study 291
Exercises 298
Summary 299
Chapter 10: Python Design Patterns I 301
The decorator pattern 301
A decorator example 302
Decorators in Python 305
The observer pattern 307
An observer example 308
The strategy pattern 310
A strategy example 311
Strategy in Python 313
The state pattern 313
A state example 314
State versus strategy 320
State transition as coroutines 320
The singleton pattern 320
Singleton implementation 321
The template pattern 325
A template example 325
Exercises 329
Summary 329
Chapter 11: Python Design Patterns II 331
The adapter pattern 331
The facade pattern 335
The flyweight pattern 337
The command pattern 341
The abstract factory pattern 346
The composite pattern 351
Exercises 355
Summary 356
Chapter 12: Testing Object-oriented Programs 357
Why test? 357
Test-driven development 359
Unit testing 360
Assertion methods 362
Reducing boilerplate and cleaning up 364
Organizing and running tests 365
Ignoring broken tests 366
Testing with py.test 368
One way to do setup and cleanup 370
A completely different way to set up variables 373
Skipping tests with py.test 377
Imitating expensive objects 378
How much testing is enough? 382
Case study 385
Implementing it 386
Exercises 391
Summary 392
Chapter 13: Concurrency 393
Threads 394
The many problems with threads 397
Shared memory 397
The global interpreter lock 398
Thread overhead 399
Multiprocessing 399
Multiprocessing pools 401
Queues 404
The problems with multiprocessing 406
Futures 406
AsyncIO 409
AsyncIO in action 410
Reading an AsyncIO future 411
AsyncIO for networking 412
Using executors to wrap blocking code 415
Streams 417
Executors 417
Case study 418
Exercises 425
Summary 426
Index 427

e-books shop

This book introduces the terminology of the object-oriented paradigm. It focuses on object-oriented design with step-by-step examples. It guides us from simple inheritance, one of the most useful tools in the object-oriented programmer's toolbox through exception handling to design patterns, an object-oriented way of looking at object-oriented concepts.
Along the way, we'll learn to integrate the object-oriented and not-so-object-oriented aspects of the Python programming language. We will learn the complexities of string and file manipulation, emphasizing (as Python 3 does) the difference between binary and textual data.
We'll then cover the joys of unit testing, using not one, but two unit testing frameworks. Finally, we'll explore, through Python's various concurrency
paradigms, how to make objects work well together at the same time.

What you need for this book
All the examples in this book rely on the Python 3 interpreter. Make sure you are not
using Python 2.7 or earlier. At the time of writing, Python 3.4 was the latest release
of Python. Most examples will work on earlier revisions of Python 3, but you are
encouraged to use the latest version to minimize frustration.
All of the examples should run on any operating system supported by Python.
If this is not the case, please report it as a bug.
Some of the examples need a working Internet connection. You'll probably want
to have one of these for extracurricular research and debugging anyway!
In addition, some of the examples in this book rely on third-party libraries that do
not ship with Python. These are introduced within the book at the time they are
used, so you do not need to install them in advance. However, for completeness,
here is a list:
• pip
• requests
• pillow
• bitarray

Who this book is for
This book specifically targets people who are new to object-oriented programming.
It assumes you have basic Python skills. You'll learn object-oriented principles in
depth. It is particularly useful for system administrator types who have used
Python as a "glue" language and would like to improve their programming skills.
If you are familiar with object-oriented programming in other languages, then this
book will help you understand the idiomatic ways to apply your knowledge in the
Python ecosystem.

A hands-on guide to mastering mobile forensics for the iOS, Android, and the Windows Phone platforms

Rohit Tamma . Oleg Skulkin . Heather Mahalik . Satish Bommisetty

e-books shop
e-books shop
Purchase Now !
Just with Paypal

Book Details
 392 p
 File Size 
 17,694 KB
 File Type
 PDF format
 2018 Packt Publishing 

About the Authors
Rohit Tamma is a security program manager currently working with Microsoft. With over
8 years of experience in the field of security, his background spans management and
technical consulting roles in the areas of application and cloud security, mobile security,
penetration testing, and security training. Rohit has also coauthored couple of books, such
as Practical Mobile Forensics and Learning Android Forensics, which explain various ways to
perform forensics on the mobile platforms. You can contact him on Twitter at @RohitTamma.
Writing this book has been a great experience because it has taught me several things,
which could not have been otherwise possible. I would like to dedicate this book to my
parents for helping me in every possible way throughout my life.

Oleg Skulkin is a digital forensics "enthusional" (enthusiast and professional) from Russia
with more than 6 years of experience, and is currently employed by Group-IB, one of the
global leaders in preventing and investigating high-tech crimes and online fraud. He holds
a number of certifications, including GCFA, MCFE, and ACE. Oleg is a coauthor of Windows
Forensics Cookbook, and you can find his articles about different aspects of digital forensics
both in Russian and foreign magazines. Finally, he is a very active blogger, and he updates
the Cyber Forensicator blog daily.
I would like to thank my mom and wife for their support and understanding, my friend,
Igor Mikhaylov, and my teammates from Group-IB Digital Forensics Lab: Valeriy Baulin,
Sergey Nikitin, Vitaliy Trifonov, Roman Rezvuhin, Artem Artemov, Alexander Ivanov,
Alexander Simonyan, Alexey Kashtanov, Pavel Zevahin, Vladimir Martyshin, Nikita
Panov, Anastasiya Barinova, and Vesta Matveeva.

Heather Mahalik is the director of forensic engineering with ManTech CARD, where she
leads the forensic effort focusing on mobile and digital exploitation. She is a senior
instructor and author for the SANS Institute, and she is also the course leader for the
FOR585 Advanced Smartphone Forensics course. With over 15 years of experience in digital
forensics, she continues to thrive on smartphone investigations, digital forensics, forensic
course development and instruction, and research on application analysis and smartphone forensics.

Satish Bommisetty is a security analyst working for a Fortune 500 company. His primary
areas of interest include iOS forensics, iOS application security, and web application
security. He has presented at international conferences, such as ClubHACK and C0C0n. He
is also one of the core members of the Hyderabad OWASP chapter. He has identified and
disclosed vulnerabilities within the websites of Google, Facebook, Yandex, PayPal, Yahoo!,
AT&T, and more, and they are listed in their hall of fame.

About the reviewer
Igor Mikhaylov has been working as a forensics expert for 21 years. During this time, he
has attended a lot of seminars and training classes in top forensic companies and forensic
departments of government organizations. He has experience and skills in cellphones
forensics, chip-off forensics, malware forensics, and other fields. He has worked on several
thousand forensic cases.

He is the reviewer of Windows Forensics Cookbook by Oleg Skulkin and Scar de Courcier, Packt Publishing, 2017.

He is the author of Mobile Forensics Cookbook, Packt Publishing, 2017.

The exponential growth in smartphones has revolutionized several aspects of our lives.
Smartphones are one of the most quickly adopted consumer technologies in recent history.
Despite their small size, smartphones are capable of performing many tasks, such as
sending private messages and confidential emails, taking photos and videos, making online
purchases, viewing sensitive information such as medical records and salary slips,
completing banking transactions, accessing social networking sites, and managing business
tasks. Hence, a mobile device is now a huge repository of sensitive data, which could
provide a wealth of information about its owner. This has in turn led to the evolution of
mobile device forensics, a branch of digital forensics, which deals with retrieving data from
a mobile device. Today, there is a huge demand for specialized forensic experts, especially
given the fact that the data retrieved from a mobile device is court-admissible.

Mobile forensics is all about using scientific methodologies to recover data stored within a
mobile phone for legal purposes. Unlike traditional computer forensics, mobile forensics
has limitations in obtaining evidence due to rapid changes in technology and the fast-paced
evolution of mobile software. With different operating systems and a wide range of models
being released onto the market, mobile forensics has expanded over the past few years.
Specialized forensic techniques and skills are required in order to extract data under
different conditions.

This book takes you through various techniques to help you learn how to forensically
recover data from different mobile devices with the iOS, Android, and Windows Mobile
operating systems. This book also covers behind the scenes details, such as how data is
stored and what tools actually do in the background, giving you deeper knowledge on
several topics. Step-by-step instructions enable you to try forensically recovering data yourself.

The book is organized in a manner that allows you to focus independently on chapters that
are specific to your required platform.

Table of Contents
Chapter 1: Introduction to Mobile Forensics 6
Why do we need mobile forensics? 7
Mobile forensics 8
Challenges in mobile forensics 10
The mobile phone evidence extraction process 12
The evidence intake phase 13
The identification phase 14
The legal authority 14
The goals of the examination 14
The make, model, and identifying information for the device 14
Removable and external data storage 15
Other sources of potential evidence 15
The preparation phase 15
The isolation phase 16
The processing phase 16
The verification phase 16
Comparing extracted data to the handset data 17
Using multiple tools and comparing the results 17
Using hash values 17
The documenting and reporting phase 17
The presentation phase 18
The archiving phase 18
Practical mobile forensic approaches 18
Overview of mobile operating systems 19
Android 19
iOS 20
Windows Phone 20
Mobile forensic tool leveling system 20
Manual extraction 22
Logical extraction 22
Hex dump 22
Chip-off 23
Micro read 23
Data acquisition methods 24
Physical acquisition 24
Logical acquisition 24
Manual acquisition 25
Potential evidence stored on mobile phones 25
Examination and analysis 26
Rules of evidence 28
Good forensic practices 29
Securing the evidence 29
Preserving the evidence 29
Documenting the evidence and changes 30
Reporting 30
Summary 31
Chapter 2: Understanding the Internals of iOS Devices 32
iPhone models 33
Identifying the correct hardware model 33
iPhone hardware 41
iPad models 42
Understanding the iPad hardware 44
Apple Watch models 45
Understanding the Apple Watch hardware 46
The filesystem 48
The HFS Plus filesystem 48
The HFS Plus volume 49
The APFS filesystem 50
The APFS structure 51
Disk layout 52
iPhone operating system 53
The iOS architecture 54
iOS security 55
Passcodes, Touch ID, and Face ID 56
Code Signing 56
Sandboxing 56
Encryption 57
Data protection 57
Address Space Layout Randomization 57
Privilege separation 57
Stack-smashing protection 57
Data execution prevention 58
Data wipe 58
Activation Lock 58
The App Store 58
Jailbreaking 59
Summary 60
Chapter 3: Data Acquisition from iOS Devices 61
Operating modes of iOS devices 62
The normal mode 62
The recovery mode 64
DFU mode 67
Setting up the forensic environment 70
Password protection and potential bypasses 70
Logical acquisition 71
Practical logical acquisition with libimobiledevice 72
Practical logical acquisition with Belkasoft Acquisition Tool 73
Practical logical acquisition with Magnet ACQUIRE 78
Filesystem acquisition 81
Practical jailbreaking 82
Practical filesystem acquisition with Elcomsoft iOS Forensic Toolkit 83
Physical acquisition 83
Practical physical acquisition with Elcomsoft iOS Forensic Toolkit 84
Summary 87
Chapter 4: Data Acquisition from iOS Backups 88
iTunes backup 89
Creating backups with iTunes 92
Understanding the backup structure 94
info.plist 95
manifest.plist 96
status.plist 96
manifest.db 97
Extracting unencrypted backups 99
iBackup Viewer 99
iExplorer 101
BlackLight 103
Encrypted backup 105
Elcomsoft Phone Breaker 105
Working with iCloud backups 107
Extracting iCloud backups 109
Summary 110
Chapter 5: iOS Data Analysis and Recovery 111
Timestamps 112
Unix timestamps 112
Mac absolute time 113
WebKit/Chrome time 113
SQLite databases 114
Connecting to a database 115
SQLite special commands 116
Standard SQL queries 117
Accessing a database using commercial tools 117
Key artifacts – important iOS database files 121
Address book contacts 122
Address book images 124
Call history 126
SMS messages 127
Calendar events 128
Notes 129
Safari bookmarks and cache 130
Photo metadata 131
Consolidated GPS cache 132
Voicemail 133
Property lists 133
Important plist files 134
The HomeDomain plist files 135
The RootDomain plist files 136
The WirelessDomain plist files 137
The SystemPreferencesDomain plist files 137
Other important files 137
Cookies 138
Keyboard cache 139
Photos 139
Thumbnails 140
Wallpaper 140
Recordings 141
Downloaded applications 141
Apple Watch 141
Recovering deleted SQLite records 144
Summary 145
Chapter 6: iOS Forensic Tools 146
Working with Cellebrite UFED Physical Analyzer 147
Features of Cellebrite UFED Physical Analyzer 147
Advanced logical acquisition and analysis with Cellebrite UFED Physical
Analyzer 148
Working with Magnet AXIOM 156
Features of Magnet AXIOM 156
Logical acquisition and analysis with Magnet AXIOM 157
Working with Belkasoft Evidence Center 166
Features of Belkasoft Evidence Center 166
iTunes backup parsing and analysis with Belkasoft Evidence Center 167
Working with Oxygen Forensic Detective 172
Features of Oxygen Forensic Detective 172
Logical acquisition and analysis with Oxygen Forensic Detective 173
Summary 178
Chapter 7: Understanding Android 179
The evolution of Android 180
The Android model 181
The Linux kernel layer 183
The Hardware Abstraction Layer 183
Libraries 184
Dalvik virtual machine 184
Android Runtime (ART) 185
The Java API framework layer 186
The system apps layer 186
Android security 186
Secure kernel 187
The permission model 188
Application sandbox 189
Secure inter-process communication 189
Application signing 189
Security-Enhanced Linux 190
Full Disk Encryption 190
Trusted Execution Environment 191
The Android file hierarchy 191
The Android file system 194
Viewing file systems on an Android device 194
Common file systems found on Android 197
Summary 199
Chapter 8: Android Forensic Setup and Pre-Data Extraction
Techniques 200
Setting up the forensic environment for Android 201
The Android Software Development Kit 201
The Android SDK installation 202
An Android Virtual Device 204
Connecting an Android device to a workstation 208
Identifying the device cable 209
Installing the device drivers 209
Accessing the connected device 210
The Android Debug Bridge 211
USB debugging 212
Accessing the device using adb 214
Detecting connected devices 214
Killing the local adb server 214
Accessing the adb shell 214
Basic Linux commands 215
Handling an Android device 218
Screen lock bypassing techniques 219
Using adb to bypass the screen lock 220
Deleting the gesture.key file 220
Updating the settings.db file 221
Checking for the modified recovery mode and adb connection 222
Flashing a new recovery partition 222
Using automated tools 223
Using Android Device Manager 225
Smudge attack 226
Using the Forgot Password/Forgot Pattern option 227
Bypassing third-party lock screens by booting into safe mode 228
Securing the USB debugging bypass using adb keys 228
Securing the USB debugging bypass in Android 4.4.2 229
Crashing the lock screen UI in Android 5.x 230
Other techniques 231
Gaining root access 232
What is rooting? 232
Rooting an Android device 233
Root access - adb shell 236
Summary 237
Chapter 9: Android Data Extraction Techniques 238
Data extraction techniques 239
Manual data extraction 240
Logical data extraction 240
ADB pull data extraction 240
Using SQLite Browser to view the data 243
Extracting device information 244
Extracting call logs 245
Extracting SMS/MMS 246
Extracting browser history 247
Analysis of social networking/IM chats 248
ADB backup extraction 249
ADB dumpsys extraction 251
Using content providers 253
Physical data extraction 257
Imaging an Android phone 257
Imaging a memory (SD) card 261
Joint Test Action Group 262
Chip-off 264
Summary 265
Chapter 10: Android Data Analysis and Recovery 266
Analyzing an Android image 267
Autopsy 267
Adding an image to Autopsy 267
Analyzing an image using Autopsy 271
Android data recovery 272
Recovering deleted data from an external SD card 273
Recovering data deleted from internal memory 280
Recovering deleted files by parsing SQLite files 280
Recovering files using file-carving techniques 283
Recovering contacts using your Google account 287
Summary 289
Chapter 11: Android App Analysis, Malware, and Reverse Engineering 290
Analyzing Android apps 291
Facebook Android app analysis 292
WhatsApp Android app analysis 294
Skype Android app analysis 294
Gmail Android app analysis 296
Google Chrome Android app analysis 297
Reverse engineering Android apps 299
Extracting an APK file from an Android device 300
Steps to reverse engineer Android apps 302
Android malware 304
How does malware spread? 307
Identifying Android malware 308
Summary 311
Chapter 12: Windows Phone Forensics 312
Windows Phone OS 312
Security model 315
Chambers 315
Encryption 316
Capability-based model 316
App sandboxing 318
Windows Phone filesystem 318
Data acquisition 321
Commercial forensic tool acquisition methods 322
Extracting data without the use of commercial tools 325
SD card data extraction methods 328
Key artifacts for examination 332
Extracting contacts and SMS 332
Extracting call history 333
Extracting internet history 333
Summary 334
Chapter 13: Parsing Third-Party Application Files 335
Third-party application overview 336
Chat applications 337
GPS applications 339
Secure applications 340
Financial applications 341
Social networking applications 341
Encoding versus encryption 345
Application data storage 348
iOS applications 349
Android applications 350
Windows Phone applications 353
Forensic methods used to extract third-party application data 353
Commercial tools 354
Oxygen Detective 354
Magnet IEF 357
UFED Physical Analyzer 360
Open source tools 361
Autopsy 361
Other methods of extracting application data 365
Summary 366
Other Books You May Enjoy 367
Index 370

e-books shop

Who this book is for
This book is intended for forensic examiners with little or basic experience in mobile
forensics or open source solutions for mobile forensics. The book will also be useful to
computer security professionals, researchers, and anyone seeking a deeper understanding
of mobile internals. It will also come in handy for those who are trying to recover
accidentally deleted data (photos, contacts, SMS messages, and more).
Loading... Protection Status