The Cosgriff House by Christopher Polly Architect is an unassuming little gem in the back streets of Annandale in Sydney’s inner-west.