When Felix left office he left the problem of the Apostle Paul with
Festus. In other words, he left Paul in jail for a few years,
procrastinating as to what to do with the him. According to the Acts of
the Apostles, one of the very first things Festus does as soon as he
gets to Judaea is to deal with Paul's confinement. He goes to Jerusalem
and hears the case against the apostle. After about nine days, Festus
returns to the capital, Caesarea, to preside over the trial. In an
effort to appease Paul's accusers, Festus asks Paul if he wouldn't mind
going to Jerusalem to answer to the charges against him, but he wisely
declines (perhaps because he would have been killed en route). Paul then
excercises his right as a Roman citizen to appeal his trial to Nero
Caesar. Festus replies, "You have appealed to Caesar. To Caesar you will
go" (25:12). Festus, as is noted in Acts 26:31, believed that Paul was
innocent, but he couldn't release him without causing riots, and
possibly Paul's murder. It was the best result for all involved. Paul
got out of jail, Festus didn't have to stain his hands with blood, while
at the same time saving face with his other subjects, and Paul's
antagonists got him out of their province.
Other than this incident in the first few days of his procuratorship, Festus has little recorded about him. Josephus notes that there were many brigands roaming around the province when he took office, (Ant. 20.8.10) and that he eliminated many of them (War 2.14.1).
Festus, as we have seen before, had to deal with at least one fake prophet. Like Judas, who revolted during Coponius' reign, and Theudas and the Egyptian who whipped up trouble during Felix's reign, this unnamed self-proclaimed messianic figure "promised [his followers] salvation and rest from troubles, if they chose to follow him into the wilderness" (Ant. 20.8.10). Unlike with the Egyptian, in this case both the fake prophet and his followers died at the hands of the Roman troops the governor sent to quell the "rebellion".
This governor also got into a little altercation with the leaders of Jerusalem's Jewish community, because of a dispute over a wall. First, King Agrippa, a nearby client king, built a room in which he could spy into the affiars of the Temple. So these "eminent men of Jerusalem" built a wall to block the line of vision (and coincidentally it blocked the view Roman troops had used to police the area). When Festus heard about this he was furious. He demanded that the wall be torn down, so the troops could keep order in the city. The Jewish leaders, though, asked if they could go directly to Nero for arbitration. Festus granted permission. Nero sided with the Jews (against his own governor!), partly because his wife Poppaea was a "Jewish-sympathizer."
In 62 Festus died persumably because of natural causes, the only procurator to have died in office. It took a while for the next governor, Albinus, to arrive in the province, and the gap in administration provided an opportunity for the high priest Annas (apparently serving a second term) to convene the Sanhendrin (the Jewish authority's judicial body that is well known from the gospels -- it usually required a procurator's approval in order to convene) and try to have James, the brother of Jesus Christ, stoned. More details of this incident are given under my biography of Albinus.
Ancient sources: Acts 24:27, 25, 26:24-32; Antiquities 20.8.9-20.9.1; War 2.14.1.