Peter Himmelman is an observant man, in all senses of the word. After lunch at a kosher fish restaurant in the Pico-Robertson neighborhood of Los Angeles one recent afternoon, we walked to a nearby apartment that he keeps as a place to wind up or wind down, write songs or to sip tea, paint, write, relax, and to enjoy Shabbat. He took off the pork pie hat that is one of his sartorial trademarks, and placed a kippah on his head. He observed that his guest was tired and suggested we meditate.

We moved to facing chairs. He showed me some breathing exercises, and gave me a mantra on which to concentrate. The words were: "My purple shirt." I was more tired than I thought, since I was wondering why he chose that phrase, until I noticed I was wearing a purple shirt. "It can be anything," he said. The phrase he often uses when he begins to meditate consists of the Hebrew words ribbonah shel olam: Master of the universe.

"Meditation is one area where assessment thinking has no place," he said. "No one does meditation well, no one does it poorly. We've put such a premium on success and failure."

Peter Himmelman is a man of many talents and accomplishments who is known to those who have heard of him, but haven't heard him, as Bob Dylan's son-in-law. He has been playing in and with bands since sixth grade in the Minneapolis suburb of St. Louis Park. (Filmmakers Joel and Ethan Coen, Sen. Al Franken, and New York Times columnist Thomas L. Friedman are also SLP natives.) He has released a dozen rock albums since 1986—the first half dozen on major corporate record labels, others on smaller indies, others self-released—all of which have received love from critics and none of which have sold well. The only Billboard chart on which he has ever appeared is the Heatseekers chart, limited to artists who have never had an album in the top 200. But the quality of his work has never flagged, and lately he has released some of his finest work, including Imperfect World (2005) and The Mystery and the Hum (2010). There is also an intentional oddity called Flimsy (2011), a collection of spoken-word songs ranging from the absurd to the heartbreaking. His new album, The Boat That Carries Us, now available on his own Himmelsongs label, is about motion, or being in motion, by air ("33K Feet"), by car ("Green Mexican Dreams"), or in spirit ("Angels Die").

Himmelman still attracts some of the best musicians in the world to play with him. The rhythm section on the new album features Leland Sklar (who played with Carole King, James Taylor, and dozens of others) on bass, and Jim Keltner, the go-to drummer for John Lennon, George Harrison, and Ringo Starr on their solo albums and tours, not to mention frequent percussionist for Dylan, Eric Clapton, Randy Newman, Steely Dan, and a hundred others. "I find I learn something new and valuable every time we talk," the 72-year-old Keltner said in an email. "Playing music with him is very much the same, with the added bonus of his strong and versatile guitar playing, very memorable melodies, and provocative lyrics for all kinds of subject matter. The challenge is to be better than his demo, which can take awhile."

Rock stardom for Himmelman was a real possibility in the mid-1980s. Critics loved him for his unpredictable but riveting stage shows. And those of us who love to linger over well-crafted lyrics enjoyed Himmelman's language of the heart, images of struggle and joy that are by turns imaginative, erotic, and transcendent without ever degenerating into pseudo-poetry or pretentious imagery. He came of age when the introduction of the compact disc had made the record companies flush, independent labels were scoring with new wave and rap, and megastars including Bruce Springsteen, U2, Prince, Def Leppard, Michael Jackson, AC/DC, Pink Floyd, and Billy Joel made it an industry of not just million-sellers but 10-million sellers. But something happened to Peter Himmelman along the road to major label stardom: The gift of his talent and ambition was overshadowed by another, deeper gift: a Jewish spiritual awakening that coincided with the release of his first major album, This Father's Day. The blues singer Robert Johnson, according to the myth, sold his soul to the devil at a Mississippi Delta crossroads. Himmelman made another choice at a crossroads in his young life, when he was 25.

***

Growing up in a nonobservant home—the very idea of devout Judaism, in 1960s Minneapolis, he said, seemed to him like something out of a Sholem Aleichem story—Himmelman was barely beyond bar mitzvah age when he began playing in bands with R&B singer Alexander O'Neal and in an influential Minneapolis-based calypso-reggae band Shangoya. His account of about getting into this band offers a microcosmic look at the adolescent blend of talent and chutzpah that quickly propelled him to the perch of major label rock success just a few years later. "After the show I went up to the bass player and said, you guys are good, but you'd be a lot better with me. He laughed, but he took my number." Seven months later, Himmelman was invited to a competitive audition. He was 16, at least a decade younger than most of his rivals. Using a small amp he had bought with his bar mitzvah money, he crushed the audition by playing "Guiltiness," an obscure Bob Marley song from an album called Exodus. "It was a perfect little longueur to throw on these blues licks I had been playing forever." Then came the interview: What, asked the leader, Aldric Peter Nelson, can you bring to the group?

"I said, 'You see that amplifier? When I'm up there with you guys, this thing's gonna shoot fucking flames. I'm going to take you over the edge. You're not gonna be this novelty calypso band, you're going to be headliners in clubs.'" They laughed and hired Himmelman. In a 2004 Minneapolis Star-Tribune obituary for Aldric Nelson, Himmelman was named as among a "who's who" of Twin Cities musicians "reared" by Shangoya. But in the late 1970s, before Himmelman's 21st birthday, his music was off in another direction. He was the singer and songwriter for the Sussman Lawrence Band, a new wave group whose music was described in a pithy question from the All Music Guide: "Has anyone wanted to be Elvis Costello as much as Peter Himmelman back in his days as a callow youth fronting the band Sussman Lawrence?" The band recorded its first album in 1979. During the early 1980s, its buzz spread, relocation in the New York area seemed like the right move. Sussman Lawrence's double album Pop City was released in 1984. A third album, released in 1985, was also a Sussman Lawrence project. But the leader's songs had grown so personal that it was released as a Peter Himmelman album, on the band's own Orange label imprint. It was called This Father's Day. The title song was recorded in the basement of the family home in 1983, a final Father's Day gift for his dying father.

Himmelman's father David looms large in his story. He was a "Jewish Marine," as Himmelman often describes him, who more than once dispatched local anti-Semites with his fists. With his dad around Peter felt secure, encouraged to pursue his ambitions. "It gave me a different perspective from people my age," Himmelman said of his father's death. "It increased my awareness of life being short. How well we grasp that is reflected in how well we act on that. I wanted to get married and have kids. I had a hunger to rebuild my family, shattered by the death of my dad."

When I first met Peter in 1985, he was living in Manhattan's Hell's Kitchen neighborhood, not far from Times Square. Despite the low-fi, deeply personal title song of This Father's Day, the video for another song, "Eleventh Confession," was breaking out at MTV. Island Records, home to Bob Marley, U2, Robert Palmer, and a host of other hit-makers, signed Himmelman, and eventually released This Father's Day in 1986 as his debut album. Peter came to my office at what was then Newsday's New York bureau at 1500 Broadway. We schmoozed for a while, and he invited me to join him and singer Kenny Vance, who had been a key part of Jay and the Americans, for a class a very sharp rabbi was giving in Brooklyn. I decided to pass. Himmelman went, and stayed.

The class was given—and is still taught—by Rabbi Simon Jacobson, who was one of the Lubavitcher Rebbe Menachem Mendel Schneerson's "oral scribes": one who memorized each of the Rebbe's talks, which couldn't be written down or recorded on Shabbat or holidays. The talks, often in Yiddish or Hebrew, had to be memorized and translated as soon as allowed. "Peter was a tall, lanky guy from Minnesota, funny, a smart-alec and skeptic," Jacobson recalled during a visit in his home the day after the gimmel tammuz, the Rebbe's 20th yahrtzeit. The house is a short walk from Chabad world headquarters on Brooklyn's Eastern Parkway. Around a large table there were bookcases with the well-worn Jewish texts familiar as a backdrop to Jacobson's many video talks and classes. There are also video cameras, microphones, and laptop computer, the tools of modern outreach.

Jacobson described the Himmelman he met in 1985 as one who had seen the hypocrisy of the Judaism he had grown up with, being concerned more with paying for the synagogue's wall-to-wall carpeting than with spiritual matters. Jacobson's classes are more concerned with discussing Judaism in terms of purpose, "the gamut of existential and metaphysical subjects," as he puts it. Himmelman stayed long into the night talking to Jacobson, who spoke about the Rebbe's wisdom and mystical abilities. "I felt comfortable with Simon immediately," Himmelman says. "He was witty and sharp and saw I was not antagonistic to Judaism. And because my dad died I was very receptive. But I still thought it was a cult." Both Jacobson and Himmelman recalled the same decisive moment in their conversations.

"Can the Rebbe do anything?" Himmelman asked. "Can he fly?"

"I've never seen the Rebbe fly," Jacobson replied. "But for the Rebbe, walking on the earth is as miraculous as flying."

Jacobson continues: "Peter seemed to appreciate that miracle and he started coming to the class on a weekly basis."

Himmelman, back in his Santa Monica home, offered his response to that story. He snapped his fingers and said: "I was in." A few weeks after their first meeting, Himmelman told Jacobson he had started lighting Shabbat candles. It was Jacobson's impression that Himmelman was making the candlesticks from broken beer bottles he found on the street in Hell's Kitchen. Not exactly, Himmelman said. "I do recall using some beer cans for Shabbat candlesticks when we were opening for Gregg Allman," he said. "Gregg was curious about their purpose."

Soon, Himmelman would be "all-in" to observant Judaism. "I had emerged from this long, sad, fallow period," he said. "A lot of grief. And it [Judaism] lifted my head out of it. It meant a lot then, and it means a lot today." While the timing was perfect for Himmelman's sense of inner peace and self-understanding, observant Judaism was not especially conducive to the work of being a rock star in the 1980s. Careers were built on touring, even when radio airplay, MTV, marketing, and promotion were still a factor, and everyone except shoplifters paid for the recorded music to which they listened.

"He turned down many great opportunities because he would not perform on Shabbos or travel on weekends," said Janet Kleinbaum, a video producer who was a top PR executive at Island through the Himmelman years and had known him since junior high. "To be honest, it frustrated and probably confused the label, but ultimately Peter was in charge of his career path, and we did the best we could to support him and promote the albums within his guidelines."

Himmelman was also marrying and starting a family in those late 1980s Island Records years. When he announced that he would marry Bob Dylan's daughter Maria, he recalled, "Rolling Stone wanted to do a big spread on the wedding: 'Bob Dylan's daughter gets married,' and we turned it down. People magazine wanted to do this whole thing, come into the house. When I turned down People, Island said, 'What are you doing? We want you to light up like a toaster.' Something like two or three invitations to be on the Tonight Show we turned down. They wondered, who is this guy, straddling the fence? Why doesn't he just give it up to the machine?" While his record company fretted, Himmelman would stand in line "for dollars, many times, and receive blessings," from the Lubavitcher Rebbe, who handed out dollar bills to the faithful for many years. "Maria and I saw him briefly in his home once, just after his wife had died," he recalled. "At the time he was seeing newlyweds, or soon-to-be-marrieds."

Rabbi Jacobson brought a handwritten letter from the Rebbe to the wedding. It did not do much for record sales but reinforced the sense of purpose, and the privacy that Himmelman has had to maintain no matter what the impact on his career.

Peter and Maria have four children, two boys, two girls, ages 18 to 24. I met Maria for about 15 seconds, when she came out to get Peter because he was late for their exercise class. She shook hands, said hello, smiled, very cordial. I got a glimpse, and no more. I had asked Peter if he could tell me how they met, or if she would talk on record for a few minutes and only about some recipes for a cookbook they wrote together (mostly her work, Peter said) that was a lagniappe for Kickstarter supporters of The Boat That Carries Us. She would not. "A few things differentiate her from anyone I've ever met," Himmelman said of his wife. "She has absolutely no desire whatever to be in the public eye. In almost 26 years, she never said one word, to anyone, on the record."

The conventional journalistic narrative might ask how being an observant Jew, married to the understandably reticent daughter of one of the most famous men on the planet who himself is just as notorious for guarding his own privacy, has limited his career. "There are certain guardrails that protect a marriage," Himmelman said. "Not that children are guardrails, but they serve that function." The holidays that require observance, he said, are protective of unions. Unplugging, literally and figuratively, is one of the blessings of Shabbat. And so is the benefit of not quite achieving the fame as an adult that he might have dreamt of as a boy. "Anyone who desires to be seen in the public eye has a lack, a need for special attention," he said. "I am drawn to that, but even at the apex, I was cognizant of the danger of the whole thing. I would play for thousands of people, but I'd have to shut that off before I walked in the door. Maria would hand me a screaming baby and say, 'What do you want, applause?' And I'd think, yes, I do. So, you really have to give fame more than I would be willing to give. It's like Baal, or some pagan religion. I can't give everything to that. The benefits of fame have come to me in smaller packages than I might have managed."

***

There is a video from the 1990s in which Himmelman performs on a Chabad telethon along with his father-in-law and the actor/musician Harry Dean Stanton. Himmelman is in the center, flanked by Dylan and Stanton. All three men are wearing kippot. Himmelman tells the host the name of the group is Chopped Liver. Dylan plays flute, recorder, and harmonica. They do three songs: a Yiddish Romanian tune Himmelman learned from his grandmother; a Spanish song featuring Dylan and Stanton; and a rendition of "Hava Nagila." When Himmelman doubles the tempo to the already speedy "Hava Nagila," Dylan, perhaps getting winded from all the wind instruments, appears to shoot Himmelman a look that only a family member could survive—or playfully ignore.

There were obvious levels of connection long before Dylan became family. "He was a guy with an unusual voice, Jewish, comes from Minnesota, a lot of identification. He didn't go to college, nor did I. [I thought] if he didn't, maybe I don't have to." Any songwriting tips, comments, criticism, or affirmation from Dylan about their shared profession? "It doesn't come up a lot, music, not as much as you might think. When we talk it's usually about other things. It would be like two accountants talking about numbers."

After three critically acclaimed but non-charting albums for Island - This Father's Day (1986), Gematria (1987), and Synesthesia (1989), Himmelman was signed by Sony Music's A&R Executive Vice President Michael Caplan, who helped him release three more quality albums in the early 1990s: From Strength to Strength (1991), Flown This Acid World (1992), and Skin (1994). Caplan's Epic artists also included the Allman Brothers Band and Stevie Ray Vaughan. His first post-Sony indie label Or Music orchestrated the implausible success of Hasidic reggae-rapper Matisyahu.

Around 1990, when Himmelman became available, Caplan had sufficient autonomy, or power, that he could sign Himmelman because he was a fan and hoped for the mass commercial breakout that never quite materialized. "I signed him because he was brilliant, when I could still sign people because I thought they were brilliant," Caplan said in a phone interview. "Because I was able to do what I wanted, I convinced the people who I worked for that he would be far bigger than he was." Himmelman's albums sold in the 50,000-60,000 range, which would be highly respectable now and wasn't bad for a cult artist, who fit neither mega-selling rock trend of the era: the pretty boy "hair bands" of the late 1980s, and the oncoming onslaught of grunge in the 1990s. The 1992 debut album by Epic's Pearl Jam sold well over 11 million copies.

Caplan, who grew up in an Orthodox Jewish family, also identified with Himmelman, having defied familial expectations of being a lawyer, doctor, or accountant. And he loved the audacity of Himmelman's concerts. "As a performer he flew without a net, and sometimes he'd fall," Caplan said, describing an acoustic showcase for Epic execs that Himmelman powered up by inviting people from the audience to play air guitar. Watching the artist acting out the arena rock ritual of jumping on amplifiers while playing silent power chords left many of Caplan's colleagues bewildered. "I loved it," Caplan said.

Caplan also identified with Himmelman's Jewish awareness, especially one night at the Manhattan live music venue the Marquee Club. "I had forgotten my father's yahrtzeit, but Peter hadn't. So, he stops his show and invites me and nine other guys up to the stage and we said Kaddish."

***

Those who profited so greatly from it mourn the passing of the business model for the recorded music business. Himmelman sees the beauty of the transition. "I don't want to be like the guy who made a fortune in the horse-and-buggy whips lamenting the era of the automobile," he said. "How was it we ever made money out of this wild and wondrous idea of trapping sound waves and charging for them in the first place?" Himmelman has also been resourceful enough to diversify, alert to recognize that when one door closes, another opens. Starting in the mid-1990s, he began recording and performing children's music. "I was always making up songs and stories for my kids when they were small," he said. "There was a story I used to tell them about this dog with amazing fur that all the kids loved to pet and how sad it was when one morning, all its hair fell out. The dog went to the vet only to learn that its hair would never grow back. Now, how and why would anyone love a hairless dog, especially since its main attribute, the wonderful fur, was gone? When the dog learned of a special polish that made its fur ultra-smooth, all the kids came running to pet it again. A happy ending!"

A woman from Minnesota with a label called Babyboom offered Himmelman $15,000 to write and record his first children's album, My Best Friend Is a Salamander, in 1997. Today, his body of work as a children's entertainer nearly rivals his major label output: five albums, including the Grammy Award-nominated My Green Kite (2007). He also became prolific writing for television, both songs and instrumental background music. He began scoring the CBS drama Judging Amy in 1999, and a song that appeared in an episode, "The Best Kind of Answer," was nominated for an Emmy in 2002. His latest work is for a new six-episode USA Network political thriller Dig, shot on location in Israel, that will air in fall 2014. Himmelman composed the dramatic underscores that range from intense and percussive to soft and ambient, Middle Eastern electronic blues very different from his singer-songwriter work. "It has a lot to do with biblical things and Jerusalem, so I don't think there's anyone better positioned in the composing world to understand the minutiae and nuances of it," he said.

Himmelman and his family have been frequent visitors to Israel. "I go there and feel some connectedness to it," he said. "Some of it is tribal, some of it mundane, some of it mystical. Every time I go I think, am I still going to be excited? Yeah." He has a "very brotherly" relationship with Israeli star David Broza, with whom he recorded a pop metal Hanukkah song, "Light Up the World" from the 1996 anthology Festival of Light. His big project these days is Big Muse, a kind of umbrella for all his creative efforts, which encourages non-musicians to write songs. He has used his songwriting exercises in myriad ways—for creativity seminars, for companies such as The Gap, institutions such as the Kellogg School of Management at Northwestern University, and the Wounded Warriors Family Project. At a Wounded Warriors gathering in 2013 in Breckenridge, Colo., he asked soldiers to write down, in 90 seconds, why they love their family members; spouses to write down why they aren't divorced from their husbands, why they still find them sexy; kids to write down why they love their dads. It was a high-risk endeavor for a self-described "Jew from Santa Monica" in front of an audience in which many were brain injured, had lost limbs, or were suffering from PTSD. The emotions were overwhelming.

"There is an adage I love from Judaism," he told me. "'What comes from the heart, enters the heart.' It's a potent statement. But you have to be damn sure when you're going out on that limb that you purify your intentions. Otherwise, you're a dead man."