Monday, October 10, 2005

Accessing Postings via HttpWebRequest/WebClient

After the post - Using CompareHTML to compare postings , few people asked me - how to access postings via WebClient.

Normally when we try to access posting via HttpWebRequest/WebClient, it will return "The remote server returned an error: (500) Internal Server Error."

The reason for the 500 server error is that authoring with MCMS is only supported with IE. And IE will always send the user agent string. So the code in WebAuthor requires this user agent string and does not check for this. As access with a user higher as subscriber is always treated as authoring the problem the above error will occur (thanks Stefan – for giving the correct reason!).

To avoid the above error when accessing a posting with a higher role than subscriber, it is required to specify the “User Agent”.

Via HttpWebRequest
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("url of the posting");
NetworkCredential networkCredential = new
NetworkCredential("username","password","domain");
httpWebRequest.Credentials = networkCredential;
httpWebRequest.UserAgent = "Mozilla/4.0+";
//UserAgent should be specified before getting the response.
HttpWebResponse webResponse = (HttpWebResponse)httpWebRequest.GetResponse();

Via WebClient
WebClient webClient = new WebClient();
NetworkCredential networkCredential = new
NetworkCredential("username","password","domain");
webClient.Credentials = networkCredential;
webClient.Headers.Add("User-Agent","Mozilla/4.0+");
//User-Agent should be specified in header before getting the response.
Stream webStream= webClient.OpenRead("url of the posting");

11 Comments:

Anonymous Stefan Goßner [MSFT] said...

MCMS does not make a difference between request using the GUID and the friendly path.
In addition 500 server errors will only occur to users with more than subscriber rights if the user agent is missing - except if your own template code has a bug that requires this.

The reason for the 500 server error is that authoring with MCMS is only supported with IE. And IE will always send the user agent string. So the code in Webauthor requires this user agent string and does not check for this.

As access with a user higher as subscriber is always treated as authoring the problem described below will happen.

Again: it does not matter if you are using the guid based or the friendly URLs.

3:03 PM  
Blogger Chester said...

Thanks for giving the correct reason. Updated!

12:59 AM  
Anonymous Björn said...

hi chester,

i had do to the same but with forms-authentication...so i made a request to the "ManualLoginSubmit.asp"-page (i had to pass some parameters like userid, pwd etc)...this will also work...if you're interested i can send you the example...

björn

5:00 PM  
Blogger Chester said...

Hi björn,

Thanks for thinking of sharing the information. You can post it here.

Cheers,
Chester.

1:56 AM  
Anonymous Anonymous said...

Hi Chester,

I am trying to do the same with forms authentication. Still i am not able to get through the login screen. Please see the code below and let me know if i am doing anything wrong.

string sGuid = Item.Cells[1].Text;
Posting posting = CmsHttpContext.Current.Searches.GetByGuid(sGuid) as Posting;

Label1.Text = Label1.Text + posting.Url;

string HTMLResult = "";
Label1.Text = "http://localhost";
Label1.Text = Label1.Text +

posting.Url;
NetworkCredential MyCredentials = new NetworkCredential("Subscriber", "Acc1234$$", "CSR");
CredentialCache Cache = new CredentialCache();

Cache.Add(new Uri(Label1.Text), "Basic", MyCredentials);
HttpWebRequest RequestHTML = (HttpWebRequest)WebRequest.Create(Label1.Text);
RequestHTML.UserAgent = "Mozilla/4.0+";
RequestHTML.Credentials = Cache;
RequestHTML.AllowAutoRedirect = true;
RequestHTML.MaximumAutomaticRedirections = 100;
HttpWebResponse HTMLResponse = (HttpWebResponse)RequestHTML.GetResponse();
StreamReader HTMLStream = new StreamReader(HTMLResponse.GetResponseStream());
HTMLResult = HTMLStream.ReadToEnd();
LabelHTML.Text = HTMLResult;
HTMLStream.Close();

Thanks,
Santhosh Kumar.P

9:24 AM  
Anonymous Anonymous said...

Hi I still having troubles with "Internal Server Error HTTP 500" when trying to GetResponse().

So far, I login into the site, then I go to the upload page, I post the data and then when I try to read the response I get the server error.

Can anyone help me with this.

webRequest.ContentLength = postData.Length;
webRequest.Timeout = 10000;

using (Stream s = webRequest.GetRequestStream())

postData.WriteTo(s);
postData.Close();
webRequest.UserAgent = "Mozilla/4.0+";

HttpWebResponse myResponse = (HttpWebResponse)webRequest.GetResponse();
myResponse.GetResponseStream();
myResponse.Close();

6:42 AM  
Blogger Zippo said...

Thank you for sharing this, I have been searching for a solution for three hours now so, thanks a lot.

For others then my problem was with this code:
m_req = (HttpWebRequest) HttpWebRequest.Create(UpdateUrl);

m_req.Credentials = new NetworkCredential("user", "password", "domain");

m_req.UserAgent = "Mozilla/4.0+";

m_req.Method = "GET";
m_req.BeginGetResponse(new AsyncCallback(ResponseReceived), null);

And the solution was to add this line as in your code:

m_req.UserAgent = "Mozilla/4.0+";

I was working against IIS 6 and for this to work you will need to set authentication method to basic authentication for the webapp you are connecting to.

Thanks a lot, regards, Sigtryggur.

9:44 AM  
Anonymous Anonymous said...

i have test harness
private static bool ResponseHasTarget(string uri,
string postData,
string target)
{
byte[] buffer = Encoding.ASCII.GetBytes(postData);

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(uri);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = buffer.Length;
req.Timeout = 5000;
//req.Credentials =CredentialCache.DefaultCredentials;
req.UserAgent = "MSIE 7.0;";
Stream reqst = req.GetRequestStream();
reqst.Write(buffer, 0, buffer.Length);

reqst.Flush();
reqst.Close();

HttpWebResponse res = (HttpWebResponse)req.GetResponse();
Stream resst = res.GetResponseStream();
StreamReader sr = new StreamReader(resst);

string result = sr.ReadToEnd();

sr.Close();
resst.Close();
if (result.IndexOf(target) >= 0)
return true;
else
return false;
}
but error:"the remote server returned an error 500 internal server error";
some body help me, if i run it on local what domain name??
please send mail to tvnhan@gmail.com thansk alot.

11:35 AM  
Anonymous Anonymous said...

There were three guys,World of warcraft gold an American and Newfoundlander.wow Power Leveling They were all going to be executed.knight noah The executioner wedding dresses said that since wedding dress designers all three were to be ms mesos executed that night, flyff money that they would each get world of warcraft power leveling to choose the flyff money method by which wedding dresses they would die Their warhammer power leveling choices were:cabal alz lethal injection,cheap aion kinah electric chair Knight Gold or world of warcraft gold by hanging.he American was afraid of needles and did'nt Dragonica crone want to be hanged. flyff money The American 2moons dil chose the wedding dresses electric chair.cdkey He sat in the chair and they wow gold pulled the flyff gold switch and nothing happened.

10:22 AM  
Blogger  said...

姫野愛MUTEKI吉野公佳鈴木早智子エロ動画YouTubeえろつべSEXエロ動画アダルトロリータMUTEKIおっぱいアダルトS1ニコニコ動画youtubeveohエロ動画ようつべエロ動画えろつべエロ動画SEXゆーちゅーぶ

東京ほっとアダルトH動画エロ動画JK・JCyoutubeエロ動画熟女ニコ動画S1エスワン無修正AV動画エロ動画ユーチューブYouTubeようつべAV無料パイパン近親相姦レイプ強姦中出し二宮沙樹パイズリ明日花キララ

1:59 PM  
Blogger  said...

オーラルセックス手こきパイパンイラマチオパイズリパイズリみひろ蒼井そら穂花原紗央莉小澤マリア初音みのりRio(柚木ティナ)秋元美由北条麻妃妃乃ひかりかすみ果穂天海つばさ藤浦めぐAV女優アダルトエロ動画JKアダルトまんこ

人妻奥様熟女熟女アダルト エロ動画マダムパンチラアダルトフェラフェラパイズリ手こき手こき痴女熟女熟女熟女熟女美脚美尻巨乳熟女巨乳

2:00 PM  

Post a Comment

<< Home