HealthCare.gov, the $630 million online insurance marketplace, was a disaster after it went live on Oct. 1, with a roster of engineering repairs that would eventually swell to more than 600 items. The private contractors who built it were pointing fingers at one another. And inside the White House, after initially saying too much traffic was to blame, Mr. Obama’s closest confidants had few good answers.

The political dangers were clear to everyone in the room: Vice President Joseph R. Biden Jr.; Kathleen Sebelius, the health secretary; Marilyn Tavenner, the Medicare chief; Denis McDonough, the chief of staff; Todd Park, the chief technology officer; and others. For 90 excruciating minutes, a furious and frustrated president peppered his team with questions, drilling into the arcane minutiae of web design as he struggled to understand the scope of a crisis that suddenly threatened his presidency.

“We created this problem we didn’t need to create,” Mr. Obama said, according to one adviser who, like several interviewed, insisted on anonymity to share details of the private session. “And it’s of our own doing, and it’s our most important initiative.”

Out of that tense Oval Office meeting grew a frantic effort aimed at rescuing not only the insurance portal and Mr. Obama’s credibility, but also the Democratic philosophy that an activist government can solve big, complex social problems. Today, that rescue effort is far from complete.

The website, which the administration promised would “function smoothly” for most people by Nov. 30, remains a work in progress. It is more stable, with many more people able to use it simultaneously than just two weeks ago. But it still suffers sporadic crashes, and large parts of the vital “back end” that processes enrollment data and transactions with insurers remain unbuilt. The president, who polls showed was now viewed by a majority of Americans as not trustworthy, has conceded that he needs to “win back” his credibility.

Another round of hardware upgrades and software fixes was planned for Saturday night. Administration officials say they will give a public update about the site’s performance on Sunday morning.

The story of how the administration confronted one of the most perilous moments in Mr. Obama’s presidency — drawn from documents and from interviews with dozens of administration officials, lawmakers, insurance executives and tech experts working inside the HealthCare.gov “war room” — reveals an insular White House that did not initially appreciate the magnitude of its self-inflicted wounds, and sought help from trusted insiders as it scrambled to protect Mr. Obama’s image.

After a month of bad publicity and intensifying Republican attacks, the sense of crisis and damage control inside the White House peaked on Oct. 30, as the president’s top aides began to fully grasp the breadth of the political challenges they faced. As Ms. Sebelius was grilled by Congressional Republicans that day, Mr. Obama flew to Boston to defend the health law and confront a new accusation: that he had lied about whether people could keep their insurance. Meanwhile, Mr. McDonough huddled at the Democratic National Committee headquarters with a small group of freshman House members whose anxiety was soaring.

The day was a brutal reminder for top White House advisers that fixing the botched health care rollout would be critical to restoring their boss’s agenda and legacy. To do that, they would have to take charge of a project that, they would come to discover, had never been fully tested and was flailing in part because of the Medicare agency’s decision not to hire a “systems integrator” that could coordinate its complex parts. The White House would also have to hold together a fragile alliance of Democratic lawmakers and insurance executives.

“If we don’t do that,” one senior White House adviser recalled, “it’s a very serious threat to the success of the legislation and a very serious threat to him. We get that.”

The urgent race to fix the website — now playing out behind the locked glass doors of the closely guarded war room in Columbia, Md. — has exposed a deeply dysfunctional relationship between the Department of Health and Human Services and its technology contractors, and tensions between the White House chief of staff and senior health department officials. It strained relations between the Obama administration and the insurance industry, helped revive a Republican Party battered after the two-week government shutdown and frustrated, even infuriated, Congressional Democrats.

But as the president’s team gathered on Oct. 15 — with a budget deal finally in sight on Capitol Hill — his difficulties were only just becoming clear to the White House. As aides left the Oval Office that evening, clutching notes filled with what Mr. McDonough called “do-outs,” or assignments, political pressure was mounting.

The moment the government reopened, Mr. Obama and his image-makers knew, the news media would turn its attention to the website fiasco; at the Oct. 15 meeting, the president directed aides to make plans for him to tell the public that “yes, the website is screwed up,” one said. Within days, Republicans would have front-page evidence that the “Obamacare train wreck” they had long predicted had become a reality.

“We knew,” said Jennifer Palmieri, the White House director of communications, “that we were a little bit on borrowed time.”

The Rollout

The early reports were encouraging as HealthCare.gov opened for business on the morning of Tuesday, Oct. 1.

The long-planned federal web portal — envisioned as an online marketplace where consumers could shop for plans, compare coverage and determine whether they qualified for subsidies — was central to Mr. Obama’s promise of affordable care. (There are also 14 state-run exchanges.) On the eve of the rollout, Ms. Sebelius, a onetime Kansas governor and former insurance commissioner who had logged countless miles promoting the health law, was ebullient.

“We’re about to make some history,” she said.

The site went live around midnight, monitored by tech teams from Ms. Tavenner’s agency, the Centers for Medicare and Medicaid Services, which supervised its development. In the West Wing, Mr. Park, the technology officer, spent the night in his office keeping tabs on traffic. Later that morning, Mr. McDonough ran into Ms. Palmieri in a White House hallway.

“Did you hear?” he asked. “The traffic is really high.”

It was a relief. Mr. McDonough, a 43-year-old former national security aide and one-time high school football defensive back known for his military-speak and sports analogies, had distributed “enrollment countdown calendars” to his staff members and warned them that “no plan survives first contact.” Yet his primary concern — that customers would not come — so far appeared unfounded.

But in Herndon, Va., at the offices of CGI Federal, the American subsidiary of a Montreal-based information technology firm that built the bulk of the site, technicians were frantic. They were beginning to realize what the White House did not: that the exchange’s problems involved much more than delays caused by high traffic. Errors were popping up everywhere. Software that assigned identities to enrollees and ensured that they saw only their own personal data, known internally as the EIdM, was being quickly overwhelmed. Customers could not log in to create accounts.

Mr. Park was dispatched to help. A Harvard graduate and a son of Korean immigrants who co-founded a health information technology firm when he was 24, Mr. Park had the job of promoting innovation. Now, he and the software engineers who built the system were desperate to figure out what was wrong.

“They kept looking, looking, looking, but there wasn’t anybody moving through the system,” a person who worked on the project said.

Account creation was the province of Quality Software Services Inc., or QSSI, a company based in Columbia, Md. Its subcontractor, Oracle, flew a high-level team of software engineers to Washington. Experts disagree on what went wrong. But several said that errors in the software code written to stitch the Oracle product into the online system and improperly configured hardware trapped users in endless technological loops. It would take eight days to resolve just that one bottleneck.

Publicly, Mr. Obama had said “interest way exceeded expectations, and that’s the good news.” But in a meeting in Mr. McDonough’s office that first weekend after the start, someone asked the question on everyone’s mind: Should we just take the website down altogether for a time so it can be fixed?

No, Mr. Park said, after consulting with the engineers in Herndon — the website needs to be up to see where the problems are. One senior White House official said they briefly considered scrapping the system altogether. They decided it was fixable.

On Capitol Hill, lawmakers were consumed with another problem: the looming threat of a government default. The House Democratic Caucus gathered in the East Room of the White House on Oct. 9; Mr. Obama, participants said, vowed to hold the line with Republicans on the debt fight and assured nervous Democrats that his team would get the health portal working.

That same day, Mr. McDonough met in his office with Jeffrey D. Zients, a multimillionaire management consultant who had developed a reputation as a troubleshooter while running the Office of Management and Budget and is scheduled to become Mr. Obama’s top economic adviser in January. For weeks, aides to Ms. Sebelius had expressed frustration with Mr. McDonough, mocking his “countdown calendar,” which they viewed as an example of micromanagement.

Now the chief of staff of a White House known for its insularity was again turning inward, looking to an Obama intimate who had no involvement in the creation of the health care website for what Mr. McDonough called “independent eyes.”

A Mad Scramble

Chaos and frustration among the engineers was growing as fast in mid-October as the list of problems they were supposed to be fixing. Across the country, insurance executives were alarmed. Almost no one was buying their products.

In Herndon, as engineers tried to come to grips with repeated crashes, a host of problems were becoming apparent: inadequate capacity in its data center and sloppy computer code, partly the result of rushed work amid the rapidly changing specifications issued by the government.

The website had barely been tested before it went live, so a large number of software and hardware defects had not been uncovered. Fixing the account creation software simply exposed other problems; people still could not register to buy insurance. A system intended to handle 50,000 simultaneous users was fundamentally unstable, unable to handle even a tiny fraction of that. As few as 500 users crippled it, according to people involved.

“These are not glitches,” one insurance executive said at the time, using a word the White House had adopted. “The extent of the problems is pretty enormous. At the end of our calls, people say, ‘It’s awful, just awful.’ ”

On Sunday, Oct. 13, with many top advisers spending as much as 75 percent of their time on the website, Mr. McDonough added a nightly 7 o’clock meeting in his office to demand updates.

Later that week, after the big damage control meeting in the Oval Office, he and Ms. Sebelius went to meet with the exhausted and disheartened staff at the Medicare agency. Republicans were calling for the health secretary’s resignation; aides say she never considered it. In the car on the way back to the White House, Mr. McDonough broached the idea of having an outsider take charge.

“Look,” he remembered telling Ms. Sebelius, “we’ve always recognized that as a management technique you’d always want independent eyes if we ran into a problem. What do you think about Jeff Zients?”

Ms. Sebelius hesitated. “Let’s think about it,” she said, by Mr. McDonough’s account.

It did not take much prodding; by the end of the ride, the secretary had agreed. Within 24 hours, Mr. Zients would assume the responsibility for fixing the website, though his name would not surface publicly until the next week. He began by quietly visiting the federal agencies and contractors. He found a technical and a personnel mess.

Relations between the Centers for Medicare and Medicaid Services and its prime contractor, CGI Federal, had soured over the summer, well before the website opened on Oct. 1. Contractors responsible for different parts of the portal barely talked to one another, hoping to avoid blame. Among the contractors, rumors were swirling: CGI Federal would be fired. IBM, one of the losing bidders, would take over. The system would be scrapped; it had to be rebuilt from scratch.

Mr. Zients decided the site needed a “systems integrator,” a single company that would take charge. On Oct. 24, Ms. Tavenner put Quality Software Services in that new role — a move that, people familiar with the project say, began to resolve conflicting and contradictory directions from her agency.

The week QSSI took over, HealthCare.gov — a site Mr. Obama once promised would be as easy to shop on as Amazon.com — went dark for 10 to 12 hours, unheard of in the online business world. But the bigger problem was organizational.

“People looked like they were busy,” said Andrew Slavitt, group executive vice president for QSSI and its parent company, Optum, “but it was hard to tell what they were working on and how it fit in.”

But while the contractors were grateful to Mr. Zients for helping to create order, they saw the administration’s “tech surge” — announced by Mr. Obama in the Rose Garden a few days before QSSI took over — as mostly an exercise in public relations.

The announcement conjured images of an army of software engineers descending on the project. In fact, the surge centered on about a half-dozen people who had taken leave from various technology companies to join the effort. They included Michael Dickerson, a site reliability engineer at Google who had also worked on Mr. Obama’s campaign and now draws praise from contractors as someone who is “actually making a difference,” one said.

Even so, one person working on the project said, “Surge was probably an overstatement.”

By late October, the website’s problems had become nightly fodder for television satirists, with “Saturday Night Live” lampooning Ms. Sebelius’s disastrous appearance earlier in the month on “The Daily Show With Jon Stewart.” (During a trip to Tennessee by Ms. Sebelius on Nov. 1, a state senator would add insult to injury by presenting her with a copy of “Websites for Dummies.”)

On Oct. 30, during three and a half hours of grueling testimony before the House Energy and Commerce Committee, Ms. Sebelius apologized. In the hearing room, the HealthCare.gov home page was displayed on a large video screen. “Please try again later,” it said. The site had crashed again.

That morning, an aide to the secretary woke up and burst into tears. “We are taking arrows every day,” she said.

Insurers Grow Anxious

Karen Ignagni was also feeling the crushing weight of the website’s problems.

The longtime chief executive of America’s Health Insurance Plans, the insurers’ trade association, Ms. Ignagni is one of the most powerful lobbyists in Washington. The daughter of a Rhode Island firefighter who got her start as a health policy analyst for the A.F.L.-C.I.O., she has been alternately tangling with and supporting Mr. Obama on health care since 2009. She risked alienating some of her own members by working toward the law’s passage.

With billions of dollars at stake for their industry, insurers voiced apprehensions even before the website’s start about the lack of thorough testing, and Ms. Ignagni presented a list of ideas to the Obama administration about what to do if the website malfunctioned. But, an insurance executive briefed on the meeting said, their concerns were waved off.

In the early weeks of October, as the industry’s dire predictions came true, the ever-careful Ms. Ignagni held her tongue. But one high-profile insurance executive went public with his concern. “There’s so much wrong, you just don’t know what’s broken until you get a lot more of it fixed,” Mark Bertolini, the chief executive of Aetna, said on CNBC.

It was harsh criticism from someone who wanted the health overhaul to work. Mr. Bertolini’s working-class background and personal experiences (his son had lymphoma) had also convinced him of the need for reducing the number of uninsured. And his company, which had invested heavily in preparing for the new law, stood to benefit.

Like his counterparts, the Aetna chief executive had invested heavily in preparing for the new law, hiring hundreds of additional workers and spending tens of millions of dollars to ready his company for the new marketplace. And while other major for-profit companies, such as UnitedHealth and Cigna, have mostly shied away from the online marketplace, Aetna is an active participant, offering plans in numerous markets.

Mr. Bertolini and a dozen other insurance executives were quickly invited to a meeting at the White House. They arrived in the Roosevelt Room on Oct. 23 to find Ms. Sebelius, Mr. McDonough and Valerie Jarrett, the White House liaison to business, among others. The mood, participants said, was one of cooperation, not conflict.

“Everyone was trying to say, let’s roll up our sleeves,” said James Roosevelt Jr., a grandson of President Franklin D. Roosevelt and the chief executive of Tufts Health Plan.

But the good feelings evaporated as insurers started informing hundreds of thousands of existing customers that their plans no longer met basic, minimum standards required by the Affordable Care Act. With the website practically unusable, insurers were panicking; their customers could not log onto HealthCare.gov to buy new plans.

Customers “are not able to piece together the complete story right now,” one frustrated executive complained at the time.

Mr. Obama, meanwhile, was under assault. After years of telling Americans, “If you like your insurance plan, you can keep it,” he was being accused of lying. On the night of Oct. 28, Ms. Jarrett, one of Mr. Obama’s closest confidantes and a guardian of his personal credibility, took to Twitter to defend him — and to shift the blame.

“FACT,” she wrote. “Nothing in #Obamacare forces people out of their health plans. No change is required unless insurance companies change existing plans.”

The tweet touched a nerve; it was not the first time the Obama White House had used the insurance industry as a scapegoat. Ms. Ignagni’s members were furious. “Here it comes — we knew it would happen,” one executive recalled thinking.

The administration made amends in a very public way. Chris Jennings, a health policy veteran who closed his consulting firm in January to coordinate health care issues for Mr. Obama, wrote an opinion article in USA Today asserting that insurers were not “cutting people loose,” but rather offering better, more comprehensive coverage. “They want to keep current enrollees as well as attract millions more who are currently uninsured,” he wrote.

Even so, the relationship between the insurers and the White House was once again strained.

‘You Can Keep It’

Inside the West Wing, where junior researchers monitor Twitter and other social media, officials knew the political controversy had moved beyond the broken website. Now it was about a broken promise. But for Mr. Obama, the mounting criticism was more than political. It felt personal.

“He was uncomfortable,” one senior adviser said. He hated the idea that so many Americans had received cancellation letters from their insurance companies and were angry because “of what the president had said — that this wouldn’t have happened.”

On Oct. 30, the president flew to Boston to talk about the Affordable Care Act at an event in Faneuil Hall, the Colonial-era meeting place where Mitt Romney, then the governor of Massachusetts, signed his own health care overhaul into law in 2006.

In addition to pledging again to fix the website, Mr. Obama for the first time acknowledged that not all people would be able to keep their health insurance. “For the vast majority of people who have health insurance that works, you can keep it,” he told the crowd. “So if you’re getting one of those letters,” he advised, “just shop around in the new marketplace.”

Aides hoped the admission would cool down the controversy. But back in Washington, the president’s adversaries had other ideas.

As senior Republican lawmakers huddled in strategy sessions to take advantage of the website debacle, their constituents began sending stories about having their health insurance canceled suddenly. Their anger at the president was palpable — and usable.

Bruno Gora, a 61-year-old self-employed promotional products distributor in Henrico, Va., for one, dashed off a note to his congressman, the House Republican leader, Representative Eric Cantor. Mr. Cantor had for years been questioning Mr. Obama’s “If you like your plan, you can keep it” promise. Now there was tangible proof that the president had been wrong.

Countless letters like that formed the backbone of the new Republican battle plan. The strategists knew that HealthCare.gov would eventually be fixed; it was time, one said, “to go heavy on the broken promise.”

Senator Mary L. Landrieu of Louisiana, a conservative Democrat who faces a tough re-election campaign in 2014, was one of the first to sense the danger. She quickly drafted legislation to allow consumers to keep their existing plans, with a title that was an unmistakable slap at the president: “The Keeping the Affordable Care Act Promise Act.”

At the White House, her legislation and a similar bill written by a Republican House member set off alarms among policy aides, who feared that letting consumers keep old plans could further undermine the health care law. Keeping healthier people — those most likely to have already bought coverage — out of the new plans could potentially cause premiums to go up sharply in 2015, they said.

On Nov. 6, Ms. Landrieu and the other “2014ers” marched to the White House, where they spent two hours in the Roosevelt Room upbraiding the president and his advisers. Aides to Mr. Obama say the meeting was called, in part, to give Democrats a chance to publicly criticize the president — a message that Vice President Biden delivered to Representative Steny H. Hoyer of Maryland, the Democratic whip, in a separate meeting with several freshman Democrats.

“Just attack us,” Mr. Biden said, according to one person present. “Blame us.”

Anxious Democrats increased the pressure. Even former President Bill Clinton casually suggested in an interview on Nov. 12 that Mr. Obama should let people keep their insurance, even if it meant changing the law. And by the next Wednesday, with no change yet announced by Mr. Obama, Democratic lawmakers were in a full-blown panic.

In a closed-door meeting of the House Democratic Caucus, lawmakers excoriated David Simas and Mike Hash, two of Mr. Obama’s top health care strategists. “The administration hasn’t shown an ability to solve the problem,” one lawmaker told them. The two officials promised that the president’s team was working on a solution, and that it would come soon.

Despite lingering concerns inside the administration about the long-term impact on the health care law, the president announced his solution the next day: insurers would be allowed to renew old plans for a year. The announcement came just hours before a vote on a Republican bill to let insurers renew old policies and sell similar ones to new customers next year. Insurance executives, who had participated in lengthy conversations with Mr. Jennings and other officials, said they were unprepared for Mr. Obama’s about-face.

But the moved satisfied most Democrats. Only 39 voted with Republicans to alter the health law, far fewer than the White House had feared.

The Fix-It Operation

After Mr. Zients arrived, he and Mr. Slavitt moved the technical guts of the rescue operation to QSSI in Columbia, Md. The war room — a command center known internally as the Exchange Operation Center, or X.O.C. — takes up the fourth floor of a nondescript office building that sits next to a shopping mall, close enough for frequent food runs to Chick-fil-A or Five Guys Burgers and Fries. The fix would happen here or not at all.

Guarded by thick glass doors that required coded card keys for entry, the room is occupied around the clock, with a “bridge line” — an open speakerphone — to other technical teams in Herndon and Tysons Corner, Va. At any given moment, about two dozen engineers and programmers cluster around laptops as they tackle one weakness in the system after another.

As the political debate raged on an hour away in Washington last week, the small group of technical experts that Mr. Zients assembled in Maryland focused on a singular task: identifying and fixing the hundreds of software and hardware malfunctions that were bringing down the site and making it inaccessible.

At the outset, the team had made what officials call a very intentional decision to focus their repair effort on making HealthCare.gov work better for consumers. That has meant putting off some “back-end” fixes for insurers, who use the site to receive applications and bill the government for subsidy payments.

Amid so much publicity about having a better website by Nov. 30, the administration is expecting a new crush of visitors to HealthCare.gov, raising fears that the site will once again be overwhelmed. The immediate goal in recent days has been to double HealthCare.gov’s capacity, so that 50,000 people will be able to log on simultaneously and 800,000 can visit in a single day. To accommodate overflow, the technicians are building a “waiting room” where consumers can queue up.

There is a secretive air about the war room — it is strictly off-limits to photographers and has been closed to reporters until now. Its unofficial manager is Mr. Dickerson, an easygoing 34-year-old who goes by Mikey and has taken a leave from Google to work temporarily for QSSI.

Mr. Dickerson brought with him the experience of someone used to the intense pressure of keeping a high-profile website operational. At Google, he helped maintain the company’s advertising servers; every second they were down, the company lost money.

On a cold, rainy night last week as one of the monitors showed 9,852 users logged onto HealthCare.gov, he likened the complex work to road repairs.

“It’s very similar to what traffic engineers do,” he said. “You can add lanes to the freeway, but maybe that makes commute times better and maybe it doesn’t. If everybody backs up on the on ramp, it doesn’t matter.”

Throughout late October and November, Mr. Zients had repeated a phrase that became his mantra: HealthCare.gov would “function smoothly for the vast majority of users” by the end of November, though he was always unclear about how that would be measured. His public updates each Friday provided snapshots of their technological roller-coaster ride, with metrics about response times and error rates.

But inside the room, 16 oversize Samsung television screens offered real time data, measured in milliseconds, of problems and delays.

When the problems occur — and they still do — the command center sees them first, in charts that suddenly spike on the television monitors. The data also serves as a reality check in a hypersensitive media environment. Last month, CNN reported that HealthCare.gov had gone down again. A quick look at the screens made it clear that whatever the problem had been, it was fleeting.

Mr. Zients’s metrics, meanwhile, are improving. When the repair effort began, response time — how long it takes a page to load — averaged eight seconds; now it is less than one. The error rate — how often users are unable to click through to the next page — was 6 percent; now it is 0.75 percent. When Mr. Dickerson announced that the day had ended with no major crashes and no one who could not log in, the engineers erupted in applause.

“That’s the job,” he said. “When things break, you have to fix them.”

But even as the White House points to its progress, the administration on Wednesday said troubles with HealthCare.gov had forced it to delay, by one year, an online exchange for small business.

Other people working on the project, speaking anonymously because they are not authorized to talk to reporters, say significant challenges remain.

Some of the companies building the system opposed an early decision by the Medicare agency to use database software from a company called MarkLogic, which handles data differently from systems by companies like IBM and Oracle. Some suggest that its unfamiliar nature slowed their work. By mid-November, more than six weeks after the rollout, the MarkLogic database — essentially the website’s virtual filing cabinet and index — continued to perform below expectations, according to one person who works in the command center.

In interviews, MarkLogic’s executives faulted inadequate computing power and instability at the site’s data center, as well as the failure to properly integrate their product, problems repeatedly cited by other website vendors.

But perhaps most important, it remains unclear whether the enrollment data being transmitted to insurers is completely accurate. In a worst-case scenario, insurance executives fear that some people may not actually get enrolled in the plans they think they have chosen, or that some people may receive wrong information about the subsidies for which they are eligible.

In recent days, Mr. Zients has sought to lower expectations, telling reporters that repairs will continue — it is an “iterative process,” he likes to say — and that there will be “no magic moment when our work is complete.”

In the White House, aides to Mr. Obama know that Republican attacks will keep coming, and that a clearer assessment of the Affordable Care Act will not come until at least the end of March, when the initial sign-up period for enrollment closes. The Congressional Budget Office has projected that seven million people will have signed up for coverage by then, but so far enrollment has been slow. During October, the federal government has reported, just 106,000 people picked new health plans, a vast majority of them through state-run exchanges.

Mr. Obama, meanwhile, is trying to turn the page. After a bruising two months in Washington, he spent the early part of last week on the West Coast, talking about other priorities — the economy and an immigration overhaul — raising money for Democrats, and trying at every turn to sound upbeat.

At a closed-door fund-raiser Tuesday night at the Beverly Hills home of the basketball star Magic Johnson, Mr. Obama made only scant reference to the law that he has long hoped will define his presidency. The president, who just two weeks earlier stood before a roomful of reporters in Washington and confessed that he had “fumbled” the rollout of his biggest legislative initiative, now confined his remarks about health care to his long-running battle with Republicans.

“I’m absolutely sure we’re going to make sure this country provides affordable health care for every single American,” Mr. Obama told the donors. “And if I have to fight for another three years to make sure that happens, I will do so.”

He did not mention the website.