A raft of startups is using open-source materials in an attempt to transform learning - terrifying traditional publishers.

By Scott Olster, editor

FORTUNE -- Call it a math lesson. Why would a school pay $80 for a textbook that may quickly become irrelevant, when it could pay around $5 or less?

A cadre of so-called open-education publishers is slowly beginning to gain the trust of schools and university systems by posing that question. Using free, open-source education materials, firms like CK-12 and Boundless are building digital textbooks and learning materials (mostly for math and science) that students and teachers can use and edit as they wish. While no single outfit yet dominates, the better such offerings get, the more traditional textbook giants like Pearson (PSO), Reed Elsevier (RUK), and Cengage ought to fear for their business models.

The U.S. spends more than $7 billion every year on K-12 textbooks, according to the FCC. And college textbook prices have increased by a whopping 812% since 1978, according to the American Enterprise Institute, surpassing inflation, college tuition increases -- even the much-discussed rise in medical expenses during that time period. College students report that they pay an average of $655 a year on books and supplies, according to a 2012 report from the National Association of College Stores.

Major budget constraints on school districts and public universities have given open-source materials a shot. Open-source outfits have begun wooing traditional textbook industry talent over to their cause. Former McGraw Hill Higher Education general manager of digital products Jay Chakrapani jumped to CK-12 in February to serve as president at the nonprofit open-education tech venture.

Chakrapani, 41, had spent five years at McGraw Hill, working to build up its digital offerings and bring the legacy textbook publisher into the future. "I can tell you that the aspirations of a McGraw Hill or a Pearson are in this direction," Chakrapani says. "I think the problem is that they are basically in the model of retrofitting existing content into these [new, digital] systems, and you can only go so far before it becomes garbage in, garbage out."

MORE: New Samsung Galaxy S4? Meh.

Instead of thinking of textbooks as an assembly of different chapters of content, CK-12 has broken down each individual facet of a given subject into small parts in its digital textbooks, known as "flexbooks." This granularity gives teachers and students the ability to customize how they learn. By contrast, traditional textbooks -- even those that are digital -- offer much less versatility. "CK-12 basically looked at STEM [science, technology, engineering, and math] and broke it down into the 5,000 fundamental concepts, and they mapped them all together," Chakrapani says. "It's not about creating a textbook and every three years putting out a new edition so you can capture more revenue. It's about thinking how a student learns."

Founded in 2007 with backing from billionaire Vinod Khosla, co-founder of Sun Microsystems and VC firm Khosla Ventures, Palo Alto, Calif.-based CK-12 is led by Vinod's wife, Neeru Khosla, and is very much a Khosla family affair. In fact, Chakrapani landed his gig as president after emailing Vinod Khosla out of the blue, responding to an article Khosla had written for TechCrunch in January 2012 on the future of education. A few emails between the two eventually led to several phone calls, which led to coffee, which ultimately led to a job offer. Chakrapani says that the Khoslas want education reform to be their legacy.

In recent years, the open-source and free education movements -- two different, but related, ideas -- have picked up steam. Free educational resources -- like a university course on Coursera, for example -- may be available for students to use at no cost, but students cannot reuse, remix, or repurpose that course content however they'd like. By contrast open-source materials like CK-12's materials are not only free, but can also be freely repurposed in any way a student or teacher sees fit. Massive open online courses (MOOC) providers like Coursera, EdX, and Udacity have been recruiting more and more universities to offer courses online to anyone for free, though many schools still insist on owning their course content.

Khan Academy, which is partly open-source (its back-end platform is proprietary, but most of its content is open), has attracted millions of students. The Harvard and MIT-backed MOOC EdX recently announced that it would begin to open up parts of its back-end course platform to give organizations all over the world an opportunity to make online classes of their own.

MORE: 5 beloved tech products that were sentenced to death

At the same time, free and open-source education firms, even nonprofits like CK-12 and EdX, need to generate revenue. As is the case with many other open-source outfits, one option is to offer add-on and other support services that help schools and teachers make the best use of the education materials these firms are making available. For-profit, free MOOCs like Coursera are experimenting with several different potential revenue streams, from licensing course content to other schools, to pairing corporate recruiters with top students.

Which model will prevail is still unclear. "Philosophically it sounds great," says Khan Academy's Salman Khan, referring to open source, "but that doesn't seem where the real value-add has been. The real value-add has been people putting out the resources so you can experience them for free." One thing is certain to Khan, though: The traditional players are not up to the task to hop on to this trend. "If a publisher wanted to compete with Facebook (FB), they would charge you $20 to sign up."

Just the same, K-12 schools and universities are dabbling in both trends. Utah's department of education launched a project in January 2012 to develop and support the use of open textbooks in high school science, math, and language arts, using CK-12's math and science texts as a starting point. Instead of costing the usual $80 per book, Utah was able to procure texts for around $5, the cost of printing from Amazon's self-publishing service CreateSpace, during a two-year pilot program.

"You give this kid a $4 or $5 book, let them use it to their heart's content," says David Wiley, a professor at Brigham Young University and leader of the open textbook pilot program in Utah. "And then you go back at the end of year with teachers, see what students struggle on, and revise and improve the book. Each year, the text gets better."

In September 2012, California passed legislation to support the development of 50 open-digital textbooks for intro-level classes taught in its massive university system. And in February, Congressman George Miller (D-Calif.) introduced legislation in the House to commit $750 million to bolster digital education efforts across the U.S., including the adoption of open-educational resources in classrooms.

MORE: 7 questions to ask before joining a startup

Major publishers are also taking notice; many are not thrilled. In fact, publishers Pearson, Cengage Learning, and Macmillan Higher Education filed a lawsuit in federal court in Manhattan against open-source education publisher Boundless Learning in March 2012. The major publishers claim that Boundless violates their intellectual property by matching the "selection, coordination, and arrangement" of their materials. Boundless responded to the lawsuit in February, requesting a jury trial. "Such legal action is an attempt to stifle startups using the power of the Internet to help students save money and become better learners," wrote Boundless CEO Ariel Diaz in a recent statement on the lawsuit.

On top of legal action, open source education will have to deal with the simple fact that educational institutions are, by nature, conservative and slow to change. And the major publishers have spent decades building sales-staff armies all across the country with close relationships to school systems and educators. The traditional publishers are also experts in tackling the Byzantine state textbook approval process.

Ideally, the major publishers, the free education players, and the open source firms will end up egging each other on, upping the ante at each step of the way, and ultimately benefiting students and teachers. After all, why can't it play out like the way open-source Linux helped propel advances from Microsoft (MSFT) and Apple (AAPL)? "I think there will always be what I think is a healthy relationship between publishers and the open-source world. They push each other forward. They challenge each other. Competition is good," says Wiley.