Josh Coburn scored the opening goal of the game for Boro and then hauled them level at 2-2 after goals from Bali Mumba and Finn Azaz had fired Argyle in front.
Sam Greenwood put Middlesbrough back in front, only for Whittaker to rescue a point in a game neither side deserved to lose.
Mumba cut inside from the left and exchanged passes with Azaz to break into the Boro box after three minutes, but visiting goalkeeper Seny Dieng was quick of his line to deny Mumba as he let fly from close range.
Any initial pressure Argyle may have had was soon forgotten as Middlesbrough wrestled control.
Matt Crooks fired a warning shot in the 20th minute and although that went well wide, Boro were soon ahead.
Lukas Engel powered in a cross from the left and Coburn finished at the near post, under pressure from Lewis Gibson, rushing back to defend his line after blocking a 23rd-minute shot.
Plymouth keeper Michael Cooper made two important saves as Middlesbrough pressed for another goal.
He palmed over Isiah Jones’ rising 27th-minute drive from the right-hand side of the box but saved his best to deny Crooks, who connected perfectly with Jonny Howson’s corner to drill a side-foot volley goalwards, only to be denied by Cooper’s reaction save on the goal-line.
Argyle responded in style, with two goals arriving before half-time courtesy of the Azaz and Mumba combination again.
Azaz threaded a ball through the Boro back-line and into the box for Mumba and this time the attacking midfielder made no mistake in beating Dieng with a measured finish.
Mumba, with a deft backheel, returned the favour four minutes later. Azaz – on loan from Aston Villa – wrong-footed marker Rav van den Berg, made space and curled a beautiful right-foot shot from 20 yards into the opposite top corner.
Greenwood sent a 20-yard free-kick just over Cooper’s crossbar in the 49th minute as Boro sought a way back into the game.
Cooper was quickly called into action minutes later, saving well low down with his feet to keep out an angled drive from Greenwood after he was put in on goal by Jones’ superb pass.
Greenwood won a penalty, drawing a foul from Argyle defender Dan Scarr, just before the hour.
Coburn’s spot-kick was brilliantly saved by Cooper, diving to his left, but the Boro striker followed up to tap in and level the scores.
Buoyed by their equaliser, Middlesbrough surged ahead again in the 64th minute with a brilliant flowing move down the right.
Jones’ pass found Crooks, whose angled cutback from the byline was stroked home first time by the impressive Greenwood at the near post.
Again Argyle responded, now in torrential rain.
Kaine Kesler-Hayden was fouled on the right and Whittaker’s 25-yard free-kick skimmed through a crowded box and past the outstretched arm of Dieng.