-->
Navigation
Learning Python, Fourth Edition

Learning Python, Fourth Edition

Now pay Easier and Secure using Paypal
Price:

Read more

by Mark Lutz

Martin Evening


e-books shop
e-books shop
Purchase Now !
Just with Paypal



Book Details
 Price
 2.00 USD
 Pages
 1214 p
 File Size
 7,573 KB
 File Type
 PDF format
 ISBN
 978-0-596-15806-4
 Copyright   
 2009 Mark Lutz. 

Preface
This book provides an introduction to the Python programming language. Python is a
popular open source programming language used for both standalone programs and
scripting applications in a wide variety of domains. It is free, portable, powerful, and
remarkably easy and fun to use. Programmers from every corner of the software industry
have found Python’s focus on developer productivity and software quality to be
a strategic advantage in projects both large and small.

Whether you are new to programming or are a professional developer, this book’s goal
is to bring you quickly up to speed on the fundamentals of the core Python language.
After reading this book, you will know enough about Python to apply it in whatever
application domains you choose to explore.

By design, this book is a tutorial that focuses on the core Python language itself, rather
than specific applications of it. As such, it’s intended to serve as the first in a two-volume
set:
• Learning Python, this book, teaches Python itself.
• Programming Python, among others, shows what you can do with Python after
you’ve learned it.
That is, applications-focused books such as Programming Python pick up where this
book leaves off, exploring Python’s role in common domains such as the Web, graphical
user interfaces (GUIs), and databases. In addition, the book Python Pocket Reference
provides additional reference materials not included here, and it is designed to supplement this book.

Because of this book’s foundations focus, though, it is able to present Python fundamentals
with more depth than many programmers see when first learning the language.
And because it’s based upon a three-day Python training class with quizzes and exercises
throughout, this book serves as a self-paced introduction to the language.

About This Fourth Edition
This fourth edition of this book has changed in three ways. This edition:
• Covers both Python 3.0 and Python 2.6—it emphasizes 3.0, but notes differences in 2.6
• Includes a set of new chapters mainly targeted at advanced core-language topics
• Reorganizes some existing material and expands it with new examples for clarity
As I write this edition in 2009, Python comes in two flavors—version 3.0 is an emerging
and incompatible mutation of the language, and 2.6 retains backward compatibility
with the vast body of existing Python code. Although Python 3 is viewed as the future
of Python, Python 2 is still widely used and will be supported in parallel with Python
3 for years to come. While 3.0 is largely the same language, it runs almost no code
written for prior releases (the mutation of print from statement to function alone,
aesthetically sound as it may be, breaks nearly every Python program ever written).

This split presents a bit of a dilemma for both programmers and book authors. While
it would be easier for a book to pretend that Python 2 never existed and cover 3 only,
this would not address the needs of the large Python user base that exists today. A vast
amount of existing code was written for Python 2, and it won’t be going away any time
soon. And while newcomers to the language can focus on Python 3, anyone who must
use code written in the past needs to keep one foot in the Python 2 world today. Since
it may be years before all third-party libraries and extensions are ported to Python 3,
this fork might not be entirely temporary.

About This Book
This section underscores some important points about this book in general, regardless
of its edition number. No book addresses every possible audience, so it’s important to
understand a book’s goals up front.

This Book’s Prerequisites
There are no absolute prerequisites to speak of, really. Both true beginners and crusty
programming veterans have used this book successfully. If you are motivated to learn
Python, this text will probably work for you. In general, though, I have found that any
exposure to programming or scripting before this book can be helpful, even if not
required for every reader.
This book is designed to be an introductory-level Python text for programmers.* It may
not be an ideal text for someone who has never touched a computer before (for instance,
we’re not going to spend any time exploring what a computer is), but I haven’t made
many assumptions about your programming background or education.
On the other hand, I won’t insult readers by assuming they are “dummies,” either,
whatever that means—it’s easy to do useful things in Python, and this book will show
you how. The text occasionally contrasts Python with languages such as C, C++, Java,
and Pascal, but you can safely ignore these comparisons if you haven’t used such languages
in the past.

This Book’s Scope and Other Books
Although this book covers all the essentials of the Python language, I’ve kept its scope
narrow in the interests of speed and size. To keep things simple, this book focuses on
core concepts, uses small and self-contained examples to illustrate points, and
sometimes omits the small details that are readily available in reference manuals. Because
of that, this book is probably best described as an introduction and a steppingstone
to more advanced and complete texts.
For example, we won’t talk much about Python/C integration—a complex topic that
is nevertheless central to many Python-based systems. We also won’t talk much about
Python’s history or development processes. And popular Python applications such as
GUIs, system tools, and network scripting get only a short glance, if they are mentioned
at all. Naturally, this scope misses some of the big picture.
By and large, Python is about raising the quality bar a few notches in the scripting world.
Some of its ideas require more context than can be provided here, and I’d be remiss if
I didn’t recommend further study after you finish this book. I hope that most readers
of this book will eventually go on to gain a more complete understanding of applicationlevel
programming from other texts.
Because of its beginner’s focus, Learning Python is designed to be naturally complemented
by O’Reilly’s other Python books. For instance, Programming Python, another
book I authored, provides larger and more complete examples, along with tutorials on
application programming techniques, and was explicitly designed to be a follow-up
text to the one you are reading now. Roughly, the current editions of Learning
Python and Programming Python reflect the two halves of their author’s training
materials—the core language, and application programming. In addition, O’Reilly’s
Python Pocket Reference serves as a quick reference supplement for looking up some
of the finer details skipped here.
Other follow-up books can also provide references, additional examples, or details
about using Python in specific domains such as the Web and GUIs. For instance,
O’Reilly’s Python in a Nutshell and Sams’s Python Essential Reference serve as useful
references, and O’Reilly’s Python Cookbook offers a library of self-contained examples
for people already familiar with application programming techniques. Because reading
books is such a subjective experience, I encourage you to browse on your own to find
advanced texts that suit your needs. Regardless of which books you choose, though,
keep in mind that the rest of the Python story requires studying examples that are more
realistic than there is space for here.
Having said that, I think you’ll find this book to be a good first text on Python, despite
its limited scope (and perhaps because of it). You’ll learn everything you need to get
started writing useful standalone Python programs and scripts. By the time you’ve finished
this book, you will have learned not only the language itself, but also how to apply
it well to your day-to-day tasks. And you’ll be equipped to tackle more advanced topics
and examples as they come your way.


Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxi
Part I. Getting Started
1. A Python Q&A Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Why Do People Use Python? 3
Software Quality 4
Developer Productivity 5
Is Python a “Scripting Language”? 5
OK, but What’s the Downside? 7
Who Uses Python Today? 7
What Can I Do with Python? 9
Systems Programming 9
GUIs 9
Internet Scripting 10
Component Integration 10
Database Programming 11
Rapid Prototyping 11
Numeric and Scientific Programming 11
Gaming, Images, Serial Ports, XML, Robots, and More 12
How Is Python Supported? 12
What Are Python’s Technical Strengths? 13
It’s Object-Oriented 13
It’s Free 13
It’s Portable 14
It’s Powerful 15
It’s Mixable 16
It’s Easy to Use 16
It’s Easy to Learn 17
It’s Named After Monty Python 17
How Does Python Stack Up to Language X? 17
Chapter Summary 18
Test Your Knowledge: Quiz 19
Test Your Knowledge: Answers 19
2. How Python Runs Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Introducing the Python Interpreter 23
Program Execution 24
The Programmer’s View 24
Python’s View 26
Execution Model Variations 29
Python Implementation Alternatives 29
Execution Optimization Tools 30
Frozen Binaries 32
Other Execution Options 33
Future Possibilities? 33
Chapter Summary 34
Test Your Knowledge: Quiz 34
Test Your Knowledge: Answers 34
3. How You Run Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
The Interactive Prompt 35
Running Code Interactively 37
Why the Interactive Prompt? 38
Using the Interactive Prompt 39
System Command Lines and Files 41
A First Script 42
Running Files with Command Lines 43
Using Command Lines and Files 44
Unix Executable Scripts (#!) 46
Clicking File Icons 47
Clicking Icons on Windows 47
The input Trick 49
Other Icon-Click Limitations 50
Module Imports and Reloads 51
The Grander Module Story: Attributes 53
import and reload Usage Notes 56
Using exec to Run Module Files 57
The IDLE User Interface 58
IDLE Basics 58
Using IDLE 60
Advanced IDLE Tools 62
Other IDEs 63
Other Launch Options 64
User-Defined Classes 101
And Everything Else 102
Chapter Summary 103
Test Your Knowledge: Quiz 103
Test Your Knowledge: Answers 104
5. Numeric Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Numeric Type Basics 105
Numeric Literals 106
Built-in Numeric Tools 108
Python Expression Operators 108
Numbers in Action 113
Variables and Basic Expressions 113
Numeric Display Formats 115
Comparisons: Normal and Chained 116
Division: Classic, Floor, and True 117
Integer Precision 121
Complex Numbers 122
Hexadecimal, Octal, and Binary Notation 122
Bitwise Operations 124
Other Built-in Numeric Tools 125
Other Numeric Types 127
Decimal Type 127
Fraction Type 129
Sets 133
Booleans 139
Numeric Extensions 140
Chapter Summary 141
Test Your Knowledge: Quiz 141
Test Your Knowledge: Answers 141
6. The Dynamic Typing Interlude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
The Case of the Missing Declaration Statements 143
Variables, Objects, and References 144
Types Live with Objects, Not Variables 145
Objects Are Garbage-Collected 146
Shared References 148
Shared References and In-Place Changes 149
Shared References and Equality 151
Dynamic Typing Is Everywhere 152
Chapter Summary 153
Test Your Knowledge: Quiz 153
Test Your Knowledge: Answers 154
7. Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
String Literals 157
Single- and Double-Quoted Strings Are the Same 158
Escape Sequences Represent Special Bytes 158
Raw Strings Suppress Escapes 161
Triple Quotes Code Multiline Block Strings 162
Strings in Action 163
Basic Operations 164
Indexing and Slicing 165
String Conversion Tools 169
Changing Strings 171
String Methods 172
String Method Examples: Changing Strings 174
String Method Examples: Parsing Text 176
Other Common String Methods in Action 177
The Original string Module (Gone in 3.0) 178
String Formatting Expressions 179
Advanced String Formatting Expressions 181
Dictionary-Based String Formatting Expressions 182
String Formatting Method Calls 183
The Basics 184
Adding Keys, Attributes, and Offsets 184
Adding Specific Formatting 185
Comparison to the % Formatting Expression 187
Why the New Format Method? 190
General Type Categories 193
Types Share Operation Sets by Categories 194
Mutable Types Can Be Changed In-Place 194
Chapter Summary 195
Test Your Knowledge: Quiz 195
Test Your Knowledge: Answers 196
8. Lists and Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Lists 197
Lists in Action 200
Basic List Operations 200
List Iteration and Comprehensions 200
Indexing, Slicing, and Matrixes 201
Changing Lists In-Place 202
Dictionaries 207
Dictionaries in Action 209
Basic Dictionary Operations 209
Changing Dictionaries In-Place 210
More Dictionary Methods 211
A Languages Table 212
Dictionary Usage Notes 213
Other Ways to Make Dictionaries 216
Dictionary Changes in Python 3.0 217
Chapter Summary 223
Test Your Knowledge: Quiz 224
Test Your Knowledge: Answers 224
9. Tuples, Files, and Everything Else . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Tuples 225
Tuples in Action 227
Why Lists and Tuples? 229
Files 229
Opening Files 230
Using Files 231
Files in Action 232
Other File Tools 238
Type Categories Revisited 239
Object Flexibility 241
References Versus Copies 241
Comparisons, Equality, and Truth 244
Python 3.0 Dictionary Comparisons 246
The Meaning of True and False in Python 246
Python’s Type Hierarchies 248
Type Objects 249
Other Types in Python 250
Built-in Type Gotchas 251
Assignment Creates References, Not Copies 251
Repetition Adds One Level Deep 252
Beware of Cyclic Data Structures 252
Immutable Types Can’t Be Changed In-Place 253
Chapter Summary 253
Test Your Knowledge: Quiz 254
Test Your Knowledge: Answers 254
Test Your Knowledge: Part II Exercises 255
Part III. Statements and Syntax
10. Introducing Python Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
Python Program Structure Revisited 261
Python’s Statements 262
A Tale of Two ifs 264
What Python Adds 264
What Python Removes 265
Why Indentation Syntax? 266
A Few Special Cases 269
A Quick Example: Interactive Loops 271
A Simple Interactive Loop 271
Doing Math on User Inputs 272
Handling Errors by Testing Inputs 273
Handling Errors with try Statements 274
Nesting Code Three Levels Deep 275
Chapter Summary 276
Test Your Knowledge: Quiz 276
Test Your Knowledge: Answers 277
11. Assignments, Expressions, and Prints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Assignment Statements 279
Assignment Statement Forms 280
Sequence Assignments 281
Extended Sequence Unpacking in Python 3.0 284
Multiple-Target Assignments 288
Augmented Assignments 289
Variable Name Rules 292
Expression Statements 295
Expression Statements and In-Place Changes 296
Print Operations 297
The Python 3.0 print Function 298
The Python 2.6 print Statement 300
Print Stream Redirection 302
Version-Neutral Printing 306
Chapter Summary 308
Test Your Knowledge: Quiz 308
Test Your Knowledge: Answers 308
12. if Tests and Syntax Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
if Statements 311
General Format 311
Basic Examples 312
Multiway Branching 312
Python Syntax Rules 314
Block Delimiters: Indentation Rules 315
Statement Delimiters: Lines and Continuations 317
A Few Special Cases 318
Truth Tests 320
The if/else Ternary Expression 321
Chapter Summary 324
Test Your Knowledge: Quiz 324
Test Your Knowledge: Answers 324
13. while and for Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
while Loops 327
General Format 328
Examples 328
break, continue, pass, and the Loop else 329
General Loop Format 329
pass 330
continue 331
break 331
Loop else 332
for Loops 334
General Format 334
Examples 335
Loop Coding Techniques 341
Counter Loops: while and range 342
Nonexhaustive Traversals: range and Slices 343
Changing Lists: range 344
Parallel Traversals: zip and map 345
Generating Both Offsets and Items: enumerate 348
Chapter Summary 349
Test Your Knowledge: Quiz 349
Test Your Knowledge: Answers 350
14. Iterations and Comprehensions, Part 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
Iterators: A First Look 351
The Iteration Protocol: File Iterators 352
Manual Iteration: iter and next 354
Other Built-in Type Iterators 356
List Comprehensions: A First Look 358
List Comprehension Basics 359
Using List Comprehensions on Files 359
Extended List Comprehension Syntax 361
Other Iteration Contexts 362
New Iterables in Python 3.0 366
The range Iterator 367
The map, zip, and filter Iterators 368
Multiple Versus Single Iterators 369
Dictionary View Iterators 370
Other Iterator Topics 372
Chapter Summary 372
Test Your Knowledge: Quiz 372
Test Your Knowledge: Answers 373
15. The Documentation Interlude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Python Documentation Sources 375
# Comments 376
The dir Function 376
Docstrings: __doc__ 377
PyDoc: The help Function 380
PyDoc: HTML Reports 383
The Standard Manual Set 386
Web Resources 387
Published Books 387
Common Coding Gotchas 387
Chapter Summary 389
Test Your Knowledge: Quiz 389
Test Your Knowledge: Answers 390
Test Your Knowledge: Part III Exercises 390
Part IV. Functions
16. Function Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
Why Use Functions? 396
Coding Functions 396
def Statements 398
def Executes at Runtime 399
A First Example: Definitions and Calls 400
Definition 400
Calls 400
Polymorphism in Python 401
A Second Example: Intersecting Sequences 402
Definition 402
Calls 403
Polymorphism Revisited 403
Local Variables 404
Chapter Summary 404
Test Your Knowledge: Quiz 405
Test Your Knowledge: Answers 405
17. Scopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
Python Scope Basics 407
Scope Rules 408
Name Resolution: The LEGB Rule 410
Scope Example 411
The Built-in Scope 412
The global Statement 414
Minimize Global Variables 415
Minimize Cross-File Changes 416
Other Ways to Access Globals 418
Scopes and Nested Functions 419
Nested Scope Details 419
Nested Scope Examples 419
The nonlocal Statement 425
nonlocal Basics 425
nonlocal in Action 426
Why nonlocal? 429
Chapter Summary 432
Test Your Knowledge: Quiz 433
Test Your Knowledge: Answers 434
18. Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
Argument-Passing Basics 435
Arguments and Shared References 436
Avoiding Mutable Argument Changes 438
Simulating Output Parameters 439
Special Argument-Matching Modes 440
The Basics 441
Matching Syntax 442
The Gritty Details 443
Keyword and Default Examples 444
Arbitrary Arguments Examples 446
Python 3.0 Keyword-Only Arguments 450
The min Wakeup Call! 453
Full Credit 454
Bonus Points 455
The Punch Line... 456
Generalized Set Functions 456
Emulating the Python 3.0 print Function 457
Using Keyword-Only Arguments 459
Chapter Summary 460
Test Your Knowledge: Quiz 461
Test Your Knowledge: Answers 462
19. Advanced Function Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
Function Design Concepts 463
Recursive Functions 465
Summation with Recursion 465
Coding Alternatives 466
Loop Statements Versus Recursion 467
Handling Arbitrary Structures 468
Function Objects: Attributes and Annotations 469
Indirect Function Calls 469
Function Introspection 470
Function Attributes 471
Function Annotations in 3.0 472
Anonymous Functions: lambda 474
lambda Basics 474
Why Use lambda? 475
How (Not) to Obfuscate Your Python Code 477
Nested lambdas and Scopes 478
Mapping Functions over Sequences: map 479
Functional Programming Tools: filter and reduce 481
Chapter Summary 483
Test Your Knowledge: Quiz 483
Test Your Knowledge: Answers 483
20. Iterations and Comprehensions, Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
List Comprehensions Revisited: Functional Tools 485
List Comprehensions Versus map 486
Adding Tests and Nested Loops: filter 487
List Comprehensions and Matrixes 489
Comprehending List Comprehensions 490
Iterators Revisited: Generators 492
Generator Functions: yield Versus return 492
Generator Expressions: Iterators Meet Comprehensions 497
Generator Functions Versus Generator Expressions 498
Generators Are Single-Iterator Objects 499
Emulating zip and map with Iteration Tools 500
Value Generation in Built-in Types and Classes 506
3.0 Comprehension Syntax Summary 507
Comprehending Set and Dictionary Comprehensions 507
Extended Comprehension Syntax for Sets and Dictionaries 508
Timing Iteration Alternatives 509
Timing Module 509
Timing Script 510
Timing Results 511
Timing Module Alternatives 513
Other Suggestions 517
Function Gotchas 518
Local Names Are Detected Statically 518
Defaults and Mutable Objects 520
Functions Without returns 522
Enclosing Scope Loop Variables 522
Chapter Summary 522
Test Your Knowledge: Quiz 523
Test Your Knowledge: Answers 523
Test Your Knowledge: Part IV Exercises 524
Part V. Modules
21. Modules: The Big Picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
Why Use Modules? 529
Python Program Architecture 530
How to Structure a Program 531
Imports and Attributes 531
Standard Library Modules 533
How Imports Work 533
1. Find It 534
2. Compile It (Maybe) 534
3. Run It 535
The Module Search Path 535
Configuring the Search Path 537
Search Path Variations 538
The sys.path List 538
Module File Selection 539
Advanced Module Selection Concepts 540
Chapter Summary 541
Test Your Knowledge: Quiz 541
Test Your Knowledge: Answers 542
22. Module Coding Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
Module Creation 543
Module Usage 544
The import Statement 544
The from Statement 545
The from * Statement 545
Imports Happen Only Once 546
import and from Are Assignments 546
Cross-File Name Changes 547
import and from Equivalence 548
Potential Pitfalls of the from Statement 548
Module Namespaces 550
Files Generate Namespaces 550
Attribute Name Qualification 552
Imports Versus Scopes 552
Namespace Nesting 553
Reloading Modules 554
reload Basics 555
reload Example 556
Chapter Summary 558
Test Your Knowledge: Quiz 558
Test Your Knowledge: Answers 558
23. Module Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561
Package Import Basics 561
Packages and Search Path Settings 562
Package __init__.py Files 563
Package Import Example 564
from Versus import with Packages 566
Why Use Package Imports? 566
A Tale of Three Systems 567
Package Relative Imports 569
Changes in Python 3.0 570
Relative Import Basics 570
Why Relative Imports? 572
The Scope of Relative Imports 574
Module Lookup Rules Summary 575
Relative Imports in Action 575
Chapter Summary 581
Test Your Knowledge: Quiz 582
Test Your Knowledge: Answers 582
24. Advanced Module Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583
Data Hiding in Modules 583
Minimizing from * Damage: _X and __all__ 584
Enabling Future Language Features 584
Mixed Usage Modes: __name__ and __main__ 585
Unit Tests with __name__ 586
Using Command-Line Arguments with __name__ 587
Changing the Module Search Path 590
The as Extension for import and from 591
Modules Are Objects: Metaprograms 591
Importing Modules by Name String 594
Transitive Module Reloads 595
Module Design Concepts 598
Module Gotchas 599
Statement Order Matters in Top-Level Code 599
from Copies Names but Doesn’t Link 600
from * Can Obscure the Meaning of Variables 601
reload May Not Impact from Imports 601
reload, from, and Interactive Testing 602
Recursive from Imports May Not Work 603
Chapter Summary 604
Test Your Knowledge: Quiz 604
Test Your Knowledge: Answers 605
Test Your Knowledge: Part V Exercises 605
Part VI. Classes and OOP
25. OOP: The Big Picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611
Why Use Classes? 612
OOP from 30,000 Feet 613
Attribute Inheritance Search 613
Classes and Instances 615
Class Method Calls 616
Coding Class Trees 616
OOP Is About Code Reuse 619
Chapter Summary 622
Test Your Knowledge: Quiz 622
Test Your Knowledge: Answers 622
26. Class Coding Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625
Classes Generate Multiple Instance Objects 625
Class Objects Provide Default Behavior 626
Instance Objects Are Concrete Items 626
A First Example 627
Classes Are Customized by Inheritance 629
A Second Example 630
Classes Are Attributes in Modules 631
Classes Can Intercept Python Operators 633
A Third Example 634
Why Use Operator Overloading? 636
The World’s Simplest Python Class 636
Classes Versus Dictionaries 639
Chapter Summary 641
Test Your Knowledge: Quiz 641
Test Your Knowledge: Answers 641
27. A More Realistic Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643
Step 1: Making Instances 644
Coding Constructors 644
Testing As You Go 645
Using Code Two Ways 646
Step 2: Adding Behavior Methods 648
Coding Methods 649
Step 3: Operator Overloading 651
Providing Print Displays 652
Step 4: Customizing Behavior by Subclassing 653
Coding Subclasses 653
Augmenting Methods: The Bad Way 654
Augmenting Methods: The Good Way 654
Polymorphism in Action 656
Inherit, Customize, and Extend 657
OOP: The Big Idea 658
Step 5: Customizing Constructors, Too 658
OOP Is Simpler Than You May Think 660
Other Ways to Combine Classes 660
Step 6: Using Introspection Tools 663
Special Class Attributes 664
A Generic Display Tool 665
Instance Versus Class Attributes 666
Name Considerations in Tool Classes 667
Our Classes’ Final Form 668
Step 7 (Final): Storing Objects in a Database 669
Pickles and Shelves 670
Storing Objects on a Shelve Database 671
Exploring Shelves Interactively 672
Updating Objects on a Shelve 674
Future Directions 675
Chapter Summary 677
Test Your Knowledge: Quiz 677
Test Your Knowledge: Answers 678
28. Class Coding Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681
The class Statement 681
General Form 681
Example 682
Methods 684
Method Example 685
Calling Superclass Constructors 686
Other Method Call Possibilities 686
Inheritance 687
Attribute Tree Construction 687
Specializing Inherited Methods 687
Class Interface Techniques 689
Abstract Superclasses 690
Python 2.6 and 3.0 Abstract Superclasses 692
Namespaces: The Whole Story 693
Simple Names: Global Unless Assigned 693
Attribute Names: Object Namespaces 693
The “Zen” of Python Namespaces: Assignments Classify Names 694
Namespace Dictionaries 696
Namespace Links 699
Documentation Strings Revisited 701
Classes Versus Modules 703
Chapter Summary 703
Test Your Knowledge: Quiz 703
Test Your Knowledge: Answers 704
29. Operator Overloading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705
The Basics 705
Constructors and Expressions: __init__ and __sub__ 706
Common Operator Overloading Methods 706
Indexing and Slicing: __getitem__ and __setitem__ 708
Intercepting Slices 708
Index Iteration: __getitem__ 710
Iterator Objects: __iter__ and __next__ 711
User-Defined Iterators 712
Multiple Iterators on One Object 714
Membership: __contains__, __iter__, and __getitem__ 716
Attribute Reference: __getattr__ and __setattr__ 718
Other Attribute Management Tools 719
Emulating Privacy for Instance Attributes: Part 1 720
String Representation: __repr__ and __str__ 721
Right-Side and In-Place Addition: __radd__ and __iadd__ 723
In-Place Addition 725
Call Expressions: __call__ 725
Function Interfaces and Callback-Based Code 727
Comparisons: __lt__, __gt__, and Others 728
The 2.6 __cmp__ Method (Removed in 3.0) 729
Boolean Tests: __bool__ and __len__ 730
Object Destruction: __del__ 732
Chapter Summary 733
Test Your Knowledge: Quiz 734
Test Your Knowledge: Answers 734
30. Designing with Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737
Python and OOP 737
Overloading by Call Signatures (or Not) 738
OOP and Inheritance: “Is-a” Relationships 739
OOP and Composition: “Has-a” Relationships 740
Stream Processors Revisited 742
OOP and Delegation: “Wrapper” Objects 745
Pseudoprivate Class Attributes 747
Name Mangling Overview 748
Why Use Pseudoprivate Attributes? 748
Methods Are Objects: Bound or Unbound 750
Unbound Methods are Functions in 3.0 752
Bound Methods and Other Callable Objects 754
Multiple Inheritance: “Mix-in” Classes 756
Coding Mix-in Display Classes 757
Classes Are Objects: Generic Object Factories 768
Why Factories? 769
Other Design-Related Topics 770
Chapter Summary 770
Test Your Knowledge: Quiz 770
Test Your Knowledge: Answers 771
31. Advanced Class Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773
Extending Built-in Types 773
Extending Types by Embedding 774
Extending Types by Subclassing 775
The “New-Style” Class Model 777
New-Style Class Changes 778
Type Model Changes 779
Diamond Inheritance Change 783
New-Style Class Extensions 788
Instance Slots 788
Class Properties 792
__getattribute__ and Descriptors 794
Metaclasses 794
Static and Class Methods 795
Why the Special Methods? 795
Static Methods in 2.6 and 3.0 796
Static Method Alternatives 798
Using Static and Class Methods 799
Counting Instances with Static Methods 800
Counting Instances with Class Methods 802
Decorators and Metaclasses: Part 1 804
Function Decorator Basics 804
A First Function Decorator Example 805
Class Decorators and Metaclasses 807
For More Details 808
Class Gotchas 808
Changing Class Attributes Can Have Side Effects 808
Changing Mutable Class Attributes Can Have Side Effects, Too 810
Multiple Inheritance: Order Matters 811
Methods, Classes, and Nested Scopes 812
Delegation-Based Classes in 3.0: __getattr__ and built-ins 814
“Overwrapping-itis” 814
Chapter Summary 815
Test Your Knowledge: Quiz 815
Test Your Knowledge: Answers 815
Test Your Knowledge: Part VI Exercises 816
Part VII. Exceptions and Tools
32. Exception Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825
Why Use Exceptions? 825
Exception Roles 826
Exceptions: The Short Story 827
Default Exception Handler 827
Catching Exceptions 828
Raising Exceptions 829
User-Defined Exceptions 830
Termination Actions 830
Chapter Summary 833
Test Your Knowledge: Quiz 833
Test Your Knowledge: Answers 833
33. Exception Coding Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835
The try/except/else Statement 835
try Statement Clauses 837
The try else Clause 839
Example: Default Behavior 840
Example: Catching Built-in Exceptions 841
The try/finally Statement 842
Example: Coding Termination Actions with try/finally 843
Unified try/except/finally 844
Unified try Statement Syntax 845
Combining finally and except by Nesting 845
Unified try Example 846
The raise Statement 848
Propagating Exceptions with raise 849
Python 3.0 Exception Chaining: raise from 849
The assert Statement 850
Example: Trapping Constraints (but Not Errors!) 851
with/as Context Managers 851
Basic Usage 852
The Context Management Protocol 853
Chapter Summary 855
Test Your Knowledge: Quiz 856
Test Your Knowledge: Answers 856
34. Exception Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 857
Exceptions: Back to the Future 858
String Exceptions Are Right Out! 858
Class-Based Exceptions 859
Coding Exceptions Classes 859
Why Exception Hierarchies? 861
Built-in Exception Classes 864
Built-in Exception Categories 865
Default Printing and State 866
Custom Print Displays 867
Custom Data and Behavior 868
Providing Exception Details 868
Providing Exception Methods 869
Chapter Summary 870
Test Your Knowledge: Quiz 871
Test Your Knowledge: Answers 871
35. Designing with Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 873
Nesting Exception Handlers 873
Example: Control-Flow Nesting 875
Example: Syntactic Nesting 875
Exception Idioms 877
Exceptions Aren’t Always Errors 877
Functions Can Signal Conditions with raise 878
Closing Files and Server Connections 878
Debugging with Outer try Statements 879
Running In-Process Tests 880
More on sys.exc_info 881
Exception Design Tips and Gotchas 882
What Should Be Wrapped 882
Catching Too Much: Avoid Empty except and Exception 883
Catching Too Little: Use Class-Based Categories 885
Core Language Summary 885
The Python Toolset 886
Development Tools for Larger Projects 887
Chapter Summary 890
Test Your Knowledge: Quiz 891
Test Your Knowledge: Answers 891
Test Your Knowledge: Part VII Exercises 891
Part VIII. Advanced Topics
36. Unicode and Byte Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895
String Changes in 3.0 896
String Basics 897
Character Encoding Schemes 897
Python’s String Types 899
Text and Binary Files 900
Python 3.0 Strings in Action 902
Literals and Basic Properties 902
Conversions 903
Coding Unicode Strings 904
Coding ASCII Text 905
Coding Non-ASCII Text 905
Encoding and Decoding Non-ASCII text 906
Other Unicode Coding Techniques 907
Converting Encodings 909
Coding Unicode Strings in Python 2.6 910
Source File Character Set Encoding Declarations 912
Using 3.0 Bytes Objects 913
Method Calls 913
Sequence Operations 914
Other Ways to Make bytes Objects 915
Mixing String Types 916
Using 3.0 (and 2.6) bytearray Objects 917
Using Text and Binary Files 920
Text File Basics 920
Text and Binary Modes in 3.0 921
Type and Content Mismatches 923
Using Unicode Files 924
Reading and Writing Unicode in 3.0 924
Handling the BOM in 3.0 926
Unicode Files in 2.6 928
Other String Tool Changes in 3.0 929
The re Pattern Matching Module 929
The struct Binary Data Module 930
The pickle Object Serialization Module 932
XML Parsing Tools 934
Chapter Summary 937
Test Your Knowledge: Quiz 937
Test Your Knowledge: Answers 937
37. Managed Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 941
Why Manage Attributes? 941
Inserting Code to Run on Attribute Access 942
Properties 943
The Basics 943
A First Example 944
Computed Attributes 945
Coding Properties with Decorators 946
Descriptors 947
The Basics 948
A First Example 950
Computed Attributes 952
Using State Information in Descriptors 953
How Properties and Descriptors Relate 955
__getattr__ and __getattribute__ 956
The Basics 957
A First Example 959
Computed Attributes 961
__getattr__ and __getattribute__ Compared 962
Management Techniques Compared 963
Intercepting Built-in Operation Attributes 966
Delegation-Based Managers Revisited 970
Example: Attribute Validations 973
Using Properties to Validate 973
Using Descriptors to Validate 975
Using __getattr__ to Validate 977
Using __getattribute__ to Validate 978
Chapter Summary 979
Test Your Knowledge: Quiz 980
Test Your Knowledge: Answers 980
38. Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 983
What’s a Decorator? 983
Managing Calls and Instances 984
Managing Functions and Classes 984
Using and Defining Decorators 984
Why Decorators? 985
The Basics 986
Function Decorators 986
Class Decorators 990
Decorator Nesting 993
Decorator Arguments 994
Decorators Manage Functions and Classes, Too 995
Coding Function Decorators 996
Tracing Calls 996
State Information Retention Options 997
Class Blunders I: Decorating Class Methods 1001
Timing Calls 1006
Adding Decorator Arguments 1008
Coding Class Decorators 1011
Singleton Classes 1011
Tracing Object Interfaces 1013
Class Blunders II: Retaining Multiple Instances 1016
Decorators Versus Manager Functions 1018
Why Decorators? (Revisited) 1019
Managing Functions and Classes Directly 1021
Example: “Private” and “Public” Attributes 1023
Implementing Private Attributes 1023
Implementation Details I 1025
Generalizing for Public Declarations, Too 1026
Implementation Details II 1029
Open Issues 1030
Python Isn’t About Control 1034
Example: Validating Function Arguments 1034
The Goal 1034
A Basic Range-Testing Decorator for Positional Arguments 1035
Generalizing for Keywords and Defaults, Too 1037
Implementation Details 1040
Open Issues 1042
Decorator Arguments Versus Function Annotations 1043
Other Applications: Type Testing (If You Insist!) 1045
Chapter Summary 1046
Test Your Knowledge: Quiz 1047
Test Your Knowledge: Answers 1047
39. Metaclasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1051
To Metaclass or Not to Metaclass 1052
Increasing Levels of Magic 1052
The Downside of “Helper” Functions 1054
Metaclasses Versus Class Decorators: Round 1 1056
The Metaclass Model 1058
Classes Are Instances of type 1058
Metaclasses Are Subclasses of Type 1061
Class Statement Protocol 1061
Declaring Metaclasses 1062
Coding Metaclasses 1063
A Basic Metaclass 1064
Customizing Construction and Initialization 1065
Other Metaclass Coding Techniques 1065
Instances Versus Inheritance 1068
Example: Adding Methods to Classes 1070
Manual Augmentation 1070
Metaclass-Based Augmentation 1071
Metaclasses Versus Class Decorators: Round 2 1073
Example: Applying Decorators to Methods 1076
Tracing with Decoration Manually 1076
Tracing with Metaclasses and Decorators 1077
Applying Any Decorator to Methods 1079
Metaclasses Versus Class Decorators: Round 3 1080
Chapter Summary 1084
Test Your Knowledge: Quiz 1084
Test Your Knowledge: Answers 1085
Part IX. Appendixes
A. Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1089
B. Solutions to End-of-Part Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1101
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1139


Bookscreen
e-books shop

About the Programs in This Book
This fourth edition of this book, and all the program examples in it, is based on Python
version 3.0. In addition, most of its examples run under Python 2.6, as described in the
text, and notes for Python 2.6 readers are mixed in along the way.
Because this text focuses on the core language, however, you can be fairly sure that
most of what it has to say won’t change very much in future releases of Python. Most
of this book applies to earlier Python versions, too, except when it does not; naturally,
if you try using extensions added after the release you’ve got, all bets are off.
As a rule of thumb, the latest Python is the best Python. Because this book focuses on
the core language, most of it also applies to Jython, the Java-based Python language
implementation, as well as other Python implementations described in Chapter 2.
Source code for the book’s examples, as well as exercise solutions, can be fetched from
the book’s website at http://www.oreilly.com/catalog/9780596158064/. So, how do you
run the examples? We’ll study startup details in Chapter 3, so please stay tuned for
information on this front.

0