When it comes to matters of the heart, Tori Roloff was her husband Zach Roloff’s first everything.

On an episode of the Little People, Big World couple’s Raising Heights podcast

released on May 23, Zach recalled his first impression of Tori after the first date

after Tori opened up about a piece of self-doubt she felt during the early stages of their courtship.

“One of my biggest insecurities dating you is, I was always kind of like, ‘Are you going to want to see what else is out there?'” she explained. “I was your first date, your first kiss, your first everything and I was always kind of in the back of my mind like one day, he’s going to wake up and he’s going to be like, ‘Oh I want to go sample something else, like I’ve had enough of this flavor. I want to go try something else.’ ”

While Zach assured her he didn’t “remember ever having [those thoughts],” he admitted that he “probably could have dated a little longer” before tying the knot.

“I was totally cool dating and you were like ‘all right, let’s get this show on the road,’ ” he recalled, joking, “I’m like, ‘Oh okay. Like already? We just started!’ ”

“That’s four years, that’s a long time!” Tori laughed. “I was waiting for it, I’m like, ‘Dude I can’t, time is ticking bro.’ Well if it wasn’t going to be with you, I was sitting there like, ‘Well hold on, I’m already 24 at this point. I got to get the show on the road.’ ”

“Thank goodness it worked out,” she added.

Zach Roloff and Tori Roloff with their kids Jackson, Lilah, and Josiah.
Tori Roloff Instagram

The couple met when Tori was working on the Roloff farm and spent the 2010 pumpkin season picking squash with Zach. They eventually began dating in 2011 and went on to get married on the family farm in July 2015. Since then, they’ve welcomed three children: Jackson, Lilah and Josiah.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

While telling PEOPLE of how they got together in 2017, Tori recalled how someone on working the farm let her know Zach was interested.

“A coworker told me, ‘Zachary really thinks you’re cute, but doesn’t think you’ll ever go out with him,’ ” she smiled. ”Our first date lasted five hours, and we’ve been together ever since.”

“Tori brought me out of my shell and made me more social,” Zach added. “She gave me confidence.”