Bob and Shirley Hansen were on a fishing trip when they woke up at 3 a.m. Monday to the sound of someone beating on the side of their boat.
Bob looked over the side and saw five people in their underwear, he told CNN affiliate KEYT-TV. Then he saw another boat in the distance, fully engulfed in flames.
"Flames probably 30 feet high. It was totally gone," he said. The first thing he did was call the Coast Guard, he said.
The 75-foot commercial diving boat, called the Conception, caught fire Labor Day off the coast of Santa Cruz Island in Southern California.
The Hansens were able to rescue the five crew members who made their way to their boat. Eight people have been found dead so far, and 26 are missing.
Shirley said some of the rescued crew members told them about the moments on the boat before the fire. One said that a 17-year-old girl on the Conception was celebrating her birthday with her parents, she said.
"It was such a hopeless, helpless feeling to watch that boat burn and know there were only five people at our boat and there was nothing we could do," she said.
She also said the five people didn't have any form of communication on them -- no keys or phones -- and she hopes they get plenty of help.
Bob told KEYT he doesn't feel he deserves to be called a hero.
"I was just a guy there in that place. I would hope that anybody would do the same thing," Bob said, adding that he wishes he could have saved everyone on board the Conception.