If this sentence is correct?

Hi Good Morning!

If you please tell me if this is correct?

Yesterday I went by Bus No.604 or I went on Bus No.604?

I would say:
Yesterday I took the bus No.604 (from XXX) to YYY.

If you went from San Diego to Los Angeles by bus, you can say Yesterday I took the bus No.604 from San Diego to Los Angeles.

I agree that ‘took’ is a better word choice than ‘went by’, though ‘went by’ is perfectly acceptable.
I would leave out the definite article from Tofu’s suggestion:

Yesterday I took bus number 604.
Yesterday I took bus number 604 (from XXX) to YYY.

Thank you for the correction, B.

If you get [color=red]a number 6 bus, it stops right in front our house.

Should [color=red]a be deleted?

No. There is more than one number 6 bus (or one bus that does the journey a number of times) and you aren’t talking about a specific one.

I know that your thinking here is because I suggested the definite article should have been left out of the earlier examples. In that case, there was also more than one number 604 bus, but the writer was referring to a specific one, so everything you have read about the articles probably suggests that ‘the’ should have been included. All I can say is that it is the natural convention of the language to leave out ‘the’ in that context.

Thank you, B.