There is a wooden path that goes alongside the house and through the National Park Service Site to Derby St. From there you can see the "Friendship" across the street and you can turn left again and walk down Derby St. to the House of the Seven Gables.
Don't miss visiting the oldest candy company in America when you are in this neighborhood. It is Ye Olde Pepper Company and it is right across the street from the "Gables". You will not want to miss either place.
Here are some photos I took when I reversed my steps on the way back earlier this week.