Go down the street

Q: Excuse me, can you tell me how to get to the castle park?

A: Sure!  Go down the street to the next corner and turn right. You will find the big roof across the street. The castle park is on your left.

Q: Thanks.

A:  Don't mention it.

Such "direction" dialogue has " go down the street " means " go straight along the street ". Some people ask me why "down"  not "up" ?

It's typical to direct the way using "go down" in spite of no slope such as up and down.

In English there is a saying "down" is used for going  far away, "up" is for coming closely.

Even if there is no slope nor stair you can use down for go straight away and up for coming to approach.

It makes sense you can use "go down "even if you direct someone to the south or  upper place. Instead you can use "up" to direct someone to come back to you.