I haven't heard much on the internet about The Man in the High Castle, save for a single bad review that I read and that I completely disagree with. The AV Club rated the pilot episode either a B or a C, which I guess is fair, because the pilot feels somewhat clunky—but pilots often do.
From the two things I've seen, most people have divided the discussion between the story and the worldbuilding. Why is this? Well, probably because the worldbuilding is phenomenal. The Greater Nazi Reich is an interesting (and horrifying) place. I don't believe I heard a single song written before 1945 in the show. There are a lot of post-1945 songs... but they're all in German, or new songs. There's no Dragnet or Car 54, but there is American Reich on TV. As in the novel, the Nazis have considerably more advanced technology than the world of our own 1962—rocket jets, for one, and lots of public transportation.
Watching Obergruppenführer Smith run New York is enthralling. The ash-fall in episode one is horrific and beautiful. What's more, the show reveals, through the alienness of the societies depicted, the deep flaws within our own. Seeing former Americans treated as second class citizens (asked to use the tradesman's door, the subject of ethnic art collecting) is a shocking and powerful statement indeed. The wedding of the already very fascist American dream of the 50s with actual fascism is a potent and heady brew. Watching a cookout on VA day in Long Island is a terrifying experience.
I don't want to say too more, except to characterize the way in which the world of the Pacific States and the Reich is built is meticulous.
Every frame is carefully designed. Every frame is beautiful. The mise-en-scene is gorgeous, the props and the design of various Nazi and Japanese architecture reflects the character of those nations perfectly (at least as they appear in the show). This sort of goes to the worldbuilding as well—San Fransisco truly looks like an Asian city. New York is a dystopian Nazi-American nightmare.
I don't want to say much about the plot because its too fun to watch it unfold. In the end, it reminded me of something by Borges—the prevalence of the I Ching and how the philosophy of the Book of Changes interacts with the story itself is very Borgesian.
Should You Watch It?
Yes. Watch it now. Watch it always. Watch the whole thing. Amazon absolutely better pick this up for a second season.