Failback and resume replication

I have two PgSQL servers, both running 9.4. Slave (B) async replicates
master (A).

If at some point, A fails and I failover to B, the slave-turned-master
creates a new timeline and can begin serving requests. However, if I'm able
to restore service on A quickly, I'd like to bring it back into service as
master (knowing that any transactions against B would be lost). If I'm okay
with those losses (or if potentially I can bring A back before any
additional transactions are committed), B still can't resume replication
because it's forked the timeline. What I'm hoping for is a way to undo that
timeline fork and allow B to resume replicating A without a full (and
lengthy) basebackup.

I know there's pg_rewind in 9.5+ but I'm not there yet. Is what I'm trying
to do possible? What are the steps for manually reverting to a slightly
early point in time on B purely so I can resume replication?

thanks!