Programing with iterators and generators

Programing with iterators and generators

By Mark Jason Dominus from Philadelphia.pm
Date: Wednesday, 23 July 2003 09:15
Duration: 180 minutes
Language:

Sometimes you'll write a function that takes too long to run because
it produces too much useful information. A search function might
locate ten thousand matches. A database query might return ten million
records. What can you do?

Perl provides a simple and familiar model for dealing with such
problems: The filehandle! Instead of reading every file in one giant
gulp, we can use filehandles to staunch the flow of information,
trickling it into the program in manageable gulps.

In this class, we'll see several important modules, such as File::Find
and DBI, which use this approach. We'll go inside these modules and
see how to implement filehandle-like data structures and objects
ourselves. We'll see how to write functions that suspend themselves
and then pick up later where they left off. We'll learn how to take
long-running slow functions and convert them into speedy
filehandle-like data generator objects.

This fundamental technique is a mainstay of programmers in other
languages, but isn't as well-known as it should be. You'll be amazed
at how many difficult problems become simple when you unleash the
power of filehandles.

Outline

Introduction:
Filehandles are Iterators
Iterators are Objects
Common Examples of Iterators: readdir, each, and DBI
Homemade iterators
Examples:
File tree scanning
Permutations
Genomic Sequence Generator
Filehandle Iterators
A Flat-File Database
Searching Databases Backwards
Random Number Generation
Alternatives to Iterators
Filters and Transforms: map and grep for iterators
Recursively-Constructed Iterators
The Semipredicate Problem
Alternative Interfaces to Iterators: Tied scalars and handles
An Extended Example: Web Spiders