Yulin began his career on stage in the 1960s, appearing in many Shakespearean plays. Moving to film and television in the 1970s, he initially appeared in such minor cinematic fare as the 1979 construction-themed drama Steel (with Art Carney). In 1983, he was cast in Scarface as a crooked cop killed by Al Pacino, and went on to appear with Pacino in the semi-documentary Looking For Richard. Other notable films include Ghostbusters II (as choleric Judge Wexler), Clear and Present Danger (as National Security Advisor James Cutter), Murder at 1600, Cradle Will Rock (as Martin Dies, chairman of the HUAC's fore-runner), Training Day, Rush Hour 2, and Fur: An Imaginary Portrait of Diane Arbus.
In television, he played both William Shakespeare and Leonardo da Vinci on Steve Allen's Meeting of the Minds, Joseph McCarthy in the TV movie Robert F. Kennedy: His Life and Times, and recurred on 24 (as another National Security Advisor, Roger Stanton) and Buffy the Vampire Slayer (as Watcher's Council head Quentin Travers). Guest appearances encompass Wonder Woman, Frasier (earning an Emmy Award nomination for playing a bellicose crime boss), Star Trek: Deep Space Nine (as Marritza in the first season episode, "Duet"), Little House on the Prairie, The X-Files, Third Watch, and Law & Order.