So I am back at the airport.
The short sojourn to Manhattan was not so bad.
A one-way ticket to NY Penn Stn is $15.
A one-way ticket to NJ Penn Stn is $7.75. From NJ Penn Stn, a PATH subway ride to the WTC site or to the NYC Penn Stn is only $1.50.
The PATH is kind of slow, less frequent, and slight confusing for travellers.
But I guess flying to NY via EWK is not as bad as I had originally thought.