(In reply to comment #0)
> Specification: http://www.w3.org/TR/html5/
> Section: http://www.whatwg.org/specs/web-apps/current-work/#top
> 
> Comment:
> HTML5 provides an opportunity to fix a long-running problem with HTTP
> Authentication.  HTTP Authentication is important, because it is the only way
> to execute a request with 100% certainty that the user has provided an
> authentication secret.

How is it more certain than with cookies?

> Furthermore, because the secret is transmitted with
> each request, it can be passed on to other authenticated services on the
> server-side.

How is the same not true of cookies?

> HTTP Authentication is often regarded as a dinosaur, but in conjunction with
> SSL, it is vastly more secure and more useful than any other authentication
> mechanism used by web sites and web applications.

How so?

> When used properly, it can
> improve security across all backend services by using credential forwarding
> instead of, for example, storing database credentials on the server where they
> could be recovered and abused by a malicious person.

How is it that you can avoid storing some form of credentials on the server?

Basically, why exactly is HTTP auth worth using instead of cookies?  You've repeatedly stated that it is, but didn't provide reasoning to support your statements.  Maybe this is all common knowledge in some circles, but your case would be a lot stronger if you state it explicitly.

I expect this problem to be solved on the browser UI level by things like the Firefox Account Manager.

(In reply to comment #1)
> How is it more certain than with cookies?
> How is the same not true of cookies?
> 

Let me try to explain further.  When cookies are used for authentication, essentially a "surrogate secret" -- namely, the Session ID or whichever cookie mechanism you are using -- is created, which is then used to authenticate the user.  That surrogate secret is then transmitted to the server on each request.  The primary reason this isn't as secure as HTTP authentication is that the surrogate secret is being stored in other places besides the user agent and the user's brain.  The session ID, for example, is stored somewhere in a data store or in a temp directory somewhere, and it could conceivably be hijacked because of that.  When the secret itself is used (as in the case of HTTP authentication), the secret does not need to be stored anywhere except in the user's brain and the user agent's memory.  On each request you know for a fact that they have provided their credentials - with a cookie, the user could merely have stolen the session ID.  There are ways to make the cookie-based authentication more secure than I have described, but it can never be as secure as when the user agent directly provides the credentials on each request.


> How so?
> How is it that you can avoid storing some form of credentials on the server?
> 

I'm glad you asked.  Consider a typical web application in which the server connects to some database, performs some logic, and allows the end user to interact with that data and logic via HTTP.  99% of all web applications written in this way will have, in some config file or maybe even hard-coded (ugh), a connection string which specifies the credentials to connect to the database.  This is not very secure, because if that code or config file is stolen, those credentials could be used by a hacker to access the database.

In the case of HTTP authentication, the application can be written in a way that it is unnecessary to store any database credentials.  If the authentication is delegated to the database itself, then the application can simply prompt for credentials over HTTP, and forward those credentials when connecting to the database.  In that way, all database credentials are stored nowhere except the users' brains.

I'm not suggesting that the standard be changed simply to accommodate an edge case like this.  I *am* suggesting that HTTP authentication is a useful but incomplete part of web development and rather than being left to die, it should receive the small bit of attention it needs to be a complete authentication solution for web applications.  What I've described in the above paragraph shouldn't be an edge case - it should be common practice, in my opinion.

Jeremy

(In reply to comment #3)
> Let me try to explain further.  When cookies are used for authentication,
> essentially a "surrogate secret" -- namely, the Session ID or whichever cookie
> mechanism you are using -- is created, which is then used to authenticate the
> user.  That surrogate secret is then transmitted to the server on each request.

Instead of transmitting the secret itself on every request.  Why is that any better?

>  The primary reason this isn't as secure as HTTP authentication is that the
> surrogate secret is being stored in other places besides the user agent and the
> user's brain.  The session ID, for example, is stored somewhere in a data store
> or in a temp directory somewhere, and it could conceivably be hijacked because
> of that.  When the secret itself is used (as in the case of HTTP
> authentication), the secret does not need to be stored anywhere except in the
> user's brain and the user agent's memory.

It, or a hash of it, must be stored on the server so that the user can be authenticated.  You could store only a hash, but you could also store only a hash of the cookie if you like.  The only difference I see is that a cookie can be invalidated without changing the underlying secret, so using a long-lived cookie means a MITM won't be able to recover the actual password unless they happen to be present on the rare occasions when it's sent.  A MITM could recover the password on every request if it's sent on every request.

> On each request you know for a fact
> that they have provided their credentials - with a cookie, the user could
> merely have stolen the session ID.

If you send the secret itself, the user could have merely stolen the secret.

> There are ways to make the cookie-based
> authentication more secure than I have described, but it can never be as secure
> as when the user agent directly provides the credentials on each request.

I really can't see any attack against cookies that can't be carried out against the system you describe by an attacker with the same capabilities.

> I'm glad you asked.  Consider a typical web application in which the server
> connects to some database, performs some logic, and allows the end user to
> interact with that data and logic via HTTP.  99% of all web applications
> written in this way will have, in some config file or maybe even hard-coded
> (ugh), a connection string which specifies the credentials to connect to the
> database.  This is not very secure, because if that code or config file is
> stolen, those credentials could be used by a hacker to access the database.

If we're talking about a typical web application as you say, anyone with access to the filesystem of the application server can almost certainly access the database server anyway, one way or another.  Web applications are typically only secured against attackers outside the local network.  (Of course, there are surely some web apps that are atypical in this respect.)

> In the case of HTTP authentication, the application can be written in a way
> that it is unnecessary to store any database credentials.  If the
> authentication is delegated to the database itself, then the application can
> simply prompt for credentials over HTTP, and forward those credentials when
> connecting to the database.  In that way, all database credentials are stored
> nowhere except the users' brains.

In what typical (your word) web application would you log into the *database* with the *user's* credentials?

> I'm not suggesting that the standard be changed simply to accommodate an edge
> case like this.  I *am* suggesting that HTTP authentication is a useful but
> incomplete part of web development and rather than being left to die, it should
> receive the small bit of attention it needs to be a complete authentication
> solution for web applications.  What I've described in the above paragraph
> shouldn't be an edge case - it should be common practice, in my opinion.

In my opinion, if an attacker gains access to your application servers, they're generally going to be able to do enough evil things that trying to prevent them from also accessing your database is fairly pointless.

(In reply to comment #4)
> 
> Instead of transmitting the secret itself on every request.  Why is that any
> better?
> 

That's what I'm trying to explain.  I guess "better" is subjective, but the fewer levels of indirection that exist between what the user agent's secret and the actual secret, the more confident you can be that you are dealing with someone who has really provided the actual secret.  If the user agent's secret is the actual secret, then you have minimized the indirection.  To rephrase: if the user agent has sent the correct credentials, there is NO WAY that the user has not provided those credentials.  If the user agent sends a session ID for a session in which someone has previously provided the correct credentials, then there IS a way that the user (of the current request) didn't provide those credentials.  That's what I mean by 100% certain.

> 
> It, or a hash of it, must be stored on the server so that the user can be
> authenticated.  You could store only a hash, but you could also store only a
> hash of the cookie if you like.  The only difference I see is that a cookie can
> be invalidated without changing the underlying secret, so using a long-lived
> cookie means a MITM won't be able to recover the actual password unless they
> happen to be present on the rare occasions when it's sent.  A MITM could
> recover the password on every request if it's sent on every request.
> 

SSL means this isn't a problem.  Unless SSL is broken, in which case we have sure got a lot of bigger worries.

> If you send the secret itself, the user could have merely stolen the secret.

Sure, but we can't really consider it our responsibility to prevent the user from sharing their credentials.  But we can do everything in our power to make sure that someone else isn't able to access the application as Joe User without his involvement.

> I really can't see any attack against cookies that can't be carried out against
> the system you describe by an attacker with the same capabilities.
> 

> 
> If we're talking about a typical web application as you say, anyone with access
> to the filesystem of the application server can almost certainly access the
> database server anyway, one way or another.  Web applications are typically
> only secured against attackers outside the local network.  (Of course, there
> are surely some web apps that are atypical in this respect.)
> 

This is exactly my point.  This is a matter of opinion, but I think that what you describe above is somewhat irresponsible.  If you have important data that should not be revealed to people that don't have valid credentials to your application, then it is not sufficient to secure only against the internet.  Each tier of the application must also be secured against the others.

> 
> In what typical (your word) web application would you log into the *database*
> with the *user's* credentials?
> 

Why do you suppose that user credentials and database credentials must be distinct?  If an application user is also a database user (which makes logical sense, because in a way your application is merely a front-end for the database), then each user can even be assigned the appropriate privileges in the database to prevent that user from harming or stealing the data of other users.  Now if Joe User does become the victim of social engineering, the hacker that stole his credentials will only be able to harm Joe User and not my entire userbase.  And maybe Joe will learn something about trusting e-mails from the Official European Lottery.

> 
> In my opinion, if an attacker gains access to your application servers, they're
> generally going to be able to do enough evil things that trying to prevent them
> from also accessing your database is fairly pointless.

Again, this is a matter of opinion, and we are now decidedly off-topic in discussing security best practices rather than the HTML5 specification.  But if a hacker somehow gains read access to my code or configuration (which is not outside the realm of possibility), I don't think he should be able to do evil things or log into my database.  If the hacker gets write access to my code, then yes, I am pretty well hosed.  But it's the developer's responsibility to close as many of these doors as possible, and connection strings are one such door (and sessions stored on disk are another).

Since we're getting off-topic, let me just summarize the purpose of my proposal:
1. HTTP authentication is part of the HTTP standard and is fully RESTful, which session-based authentication is not.
2. I assert that HTTP authentication is useful and engenders good security practices, though not everyone agrees with this assertion.
3. HTTP authentication is missing a few key features which could be addressed in the HTML5 standard, which would make it nearly perfect.  This could be accomplished without changing the existing HTTP authentication spec.

Now, at the risk of further debate, let me ask: If by some miracle my proposal were accepted, and a year from now it were easily possible to use HTTP authentication (either basic or digest) and retain your nice-looking login forms and logout buttons, would you continue to use session-based authentication instead?  If so, why? You would probably be able to eliminate sessions entirely from a large portion of your work and have fully RESTful web applications.  What is it about cookies that make them better for authentication other than the fact that they currently have the features that would be addressed by my proposal?

Jeremy

Okay, well, I'm approximately 98% sure that this bug is going to be closed WONTFIX anyway.  But at the risk of continuing a pointless argument (http://xkcd.com/386/):

(In reply to comment #5)
> That's what I'm trying to explain.  I guess "better" is subjective, but the
> fewer levels of indirection that exist between what the user agent's secret and
> the actual secret, the more confident you can be that you are dealing with
> someone who has really provided the actual secret.

Your threat model is wrong.  Your goal is not to deal with a party that provided the actual secret -- your goal is to deal with the genuine user.  If your protocol provides the actual secret to adversaries, like by broadcasting it in plaintext on every HTTP request, that is a security vulnerability.  You can't deal with all cases where the adversary learns the password, but you can avoid exposing it yourself.

> SSL means this isn't a problem.  Unless SSL is broken, in which case we have
> sure got a lot of bigger worries.

The overwhelming majority of websites do not use SSL, nor will they use SSL in the foreseeable future.  It's all very well for academics to assume everyone uses SSL, but HTML5 is concerned with the real world, not academics.

> This is exactly my point.  This is a matter of opinion, but I think that what
> you describe above is somewhat irresponsible.  If you have important data that
> should not be revealed to people that don't have valid credentials to your
> application, then it is not sufficient to secure only against the internet. 
> Each tier of the application must also be secured against the others.

However, this is not how almost anyone writes applications in real life.  Again, HTML5 is about real-world use-cases, not theory.

> Why do you suppose that user credentials and database credentials must be
> distinct?

Because I've never heard of any real-world web application where they're the same.  At least in MySQL, it would be completely ridiculous -- it wouldn't scale to the necessary number of users and permissions aren't granular enough to be useful.  Again, this is about real-world use-cases, not theoretical worlds where everyone writes software in some idealized modular way.

> If an application user is also a database user (which makes logical
> sense, because in a way your application is merely a front-end for the
> database), then each user can even be assigned the appropriate privileges in
> the database to prevent that user from harming or stealing the data of other
> users.  Now if Joe User does become the victim of social engineering, the
> hacker that stole his credentials will only be able to harm Joe User and not my
> entire userbase.

This is normally accomplished by permissions checks within the application, not the database.

> 2. I assert that HTTP authentication is useful and engenders good security
> practices, though not everyone agrees with this assertion.

Because you haven't provided any real-world use-cases.  You've only talked about theoretical scenarios, no actual real applications that people are using that would benefit from this change.  Nobody is going to waste time implementing HTML features unless it's likely that they'll be *widely* used.

> Now, at the risk of further debate, let me ask: If by some miracle my proposal
> were accepted, and a year from now it were easily possible to use HTTP
> authentication (either basic or digest) and retain your nice-looking login
> forms and logout buttons, would you continue to use session-based
> authentication instead?  If so, why? You would probably be able to eliminate
> sessions entirely from a large portion of your work and have fully RESTful web
> applications.  What is it about cookies that make them better for
> authentication other than the fact that they currently have the features that
> would be addressed by my proposal?

Since I live in the real world, I write applications (e.g., MediaWiki) that are mostly going to be used over HTTP, not HTTPS.  Sending the password on every request is therefore less secure than sending it only occasionally for login.

Also, with cookies, the browser isn't forced to store the password for the user to remain logged in, only the cookie.  Thus it's more secure if the user's computer is compromised, since it reduces the chance that the password is stored on the computer for an attacker to steal.

If I were writing a quick and simple application, I'd prefer basic HTTP auth if it were nice to use, because it's simpler -- although less secure.

(In reply to comment #6)
> Okay, well, I'm approximately 98% sure that this bug is going to be closed
> WONTFIX anyway.  

Agreed.  I still think it's important to make the case, even though it has no chance of being heard.

> But at the risk of continuing a pointless argument

I disagree that it's pointless, and I see it as useful debate rather than pointless argument.  But I'll concede after this that I've no chance of getting HTTP auth fixed, because most people think like you do.

> 
> 
> Your threat model is wrong.  Your goal is not to deal with a party that
> provided the actual secret -- your goal is to deal with the genuine user.  

Yes, and getting the actual secret is the most I can do to assure I'm dealing with the genuine user.  At least I know they have the secret.

> If your protocol provides the actual secret to adversaries, like by broadcasting
> it in plaintext on every HTTP request, 

It doesn't.  I use SSL.

> 
> The overwhelming majority of websites do not use SSL, nor will they use SSL in
> the foreseeable future.

Agree to disagree.  Depends on the application.  I don't expect CNN.com to use SSL, but I sure expect my bank to use it.  Some websites have important, private information, and they are no less valid than your website.

> 
> However, this is not how almost anyone writes applications in real life. 
> Again, HTML5 is about real-world use-cases, not theory.
>

But my bank is in the real world.  My stockbroker is in the real world.  And I really hope they don't share that attitude.  I would rather hope they are as fanatical about protecting data as I am.  And it pains me to think that sitting in one of their config files somewhere are the keys for stealing all of my private financial data.
 
> At least in MySQL, it would be completely ridiculous

No comment.

> 
> This is normally accomplished by permissions checks within the application, not
> the database.
> 

Again, I respect you as a programmer, but don't assume your apps are "normal" for other sectors.  MediaWiki and similar apps don't necessarily have to fanatically protect data.  But other applications do.  PCI compliant ones, for example.

> > 2. I assert that HTTP authentication is useful and engenders good security
> > practices, though not everyone agrees with this assertion.
> 
> Because you haven't provided any real-world use-cases.

I feel like I have.  Maybe you don't agree that my use cases are real-world, but to my users they are.

> 
> Since I live in the real world, *I write applications* (e.g., MediaWiki) that are
> mostly going to be used over HTTP, not HTTPS.  Sending the password on every
> request is therefore less secure than sending it only occasionally for login.
> 

Emphasis mine.  I think this is the origin of our disagreement.  You write applications - good ones, at that - which are meant to be widely (in the case of MediaWiki, *very widely*) deployed by other people who don't necessarily need, want, or understand how to protect information.  You write those apps to take advantage of easily available dependencies (i.e. MySQL).  You write those apps assuming your users probably won't deploy SSL, which they won't.  And that's exactly what you should do, *in that situation*.  

Not everyone's situation is like that.  Other people live in the real world, and not all of them write the same apps as you.  I would argue that a lot of important websites DO need to use SSL and DO need to take fanatical measures to protect their data.  Fixing HTTP auth once and for all would provide a useful tool to that end. 

> Also, with cookies, the browser isn't forced to store the password for the user
> to remain logged in, only the cookie.  Thus it's more secure if the user's
> computer is compromised, since it reduces the chance that the password is
> stored on the computer for an attacker to steal.

Two flaws here: 

1. I have never seen an attack which steals credentials out of a user agent's memory, and if one exists, then it's not my responsibility to combat that.  It's the author of the user agent who must be on the lookout for such vulnerabilities.  If you meant that they would steal it out of the user's saved passwords, then your users' saved passwords are no less vulnerable to that attack.

2. If the attacker can steal credentials out of the UA's memory, they can just as easily steal a session cookie, or a 'log me in automatically' cookie.  And there are many more vectors for stealing a session cookie than the hypothetical one you described for stealing HTTP credentials.

> 
> If I were writing a quick and simple application, I'd prefer basic HTTP auth if
> it were nice to use, because it's simpler -- although less secure.

Again - when used properly, it is at least as secure, if not more so.  That's my story and I'm sticking to it!

I'll try to stop debating now.  Again, I know I have very little chance of being heard.  I can only hope that one of the folks behind RFC 2617 will see this big argument and think to themselves, "Yeah, HTTP auth *was* a good idea and we *should* try to perfect it."  Otherwise, I concede your point that I'm pretty much talking to myself.  I just thought it was important to try.

If you want the last word, go ahead and reply once more - I'll try *really hard* not to say anything further :)

Jeremy

(In reply to comment #7)
> Not everyone's situation is like that.  Other people live in the real world,
> and not all of them write the same apps as you.

Then it would help your case a lot if you provided specific examples of real-world (*not* hypothetical) applications that you've worked on or used, explained a specific real-world attack that could be carried out on those applications, and explained how HTTP auth would have prevented that attack.  If the application is secret or in-house, you could give a basic rundown of how it's set up, as much as is necessary to understand the attack.

> 1. I have never seen an attack which steals credentials out of a user agent's
> memory, and if one exists, then it's not my responsibility to combat that. 
> It's the author of the user agent who must be on the lookout for such
> vulnerabilities.  If you meant that they would steal it out of the user's saved
> passwords, then your users' saved passwords are no less vulnerable to that
> attack.

With cookies, there's not much need to save passwords.  Lots of sites will let you stay logged in for months or forever without having to re-enter your password.  But if you use basic HTTP auth, then you either have to re-enter your password for every site every time you start the browser, or have it save the passwords to disk.  So it will encourage password saving much more strongly.

> 2. If the attacker can steal credentials out of the UA's memory, they can just
> as easily steal a session cookie, or a 'log me in automatically' cookie.  And
> there are many more vectors for stealing a session cookie than the hypothetical
> one you described for stealing HTTP credentials.

Yes, but a stolen cookie is much less damaging.  It can have a limited lifetime, can be canceled, and doesn't give information about the password itself (which is significant when users use the same or similar passwords across sites, which most do).  A leaked password is much worse.

> I'll try to stop debating now.

I actually think we're making progress.

(In reply to comment #8)
> 
> > I'll try to stop debating now.
> 
> I actually think we're making progress.

You do?  I'm happy to keep up the debate; I'm just worried about the half-dozen people who get CC'ed every time we post a reply.


> Then it would help your case a lot if you provided specific examples of
> real-world (*not* hypothetical) applications

You keep saying my examples are hypothetical, and I keep saying they're not.  I use HTTP authentication right now in several of my applications.  I can't really show you, because you're not one of my users.  Sorry.  I have worked out a complicated system of JavaScript to let me use a login form for HTTP auth in most browsers, and for browsers that don't support that (webkit, Opera) the users are prompted for their credentials using the UA's native dialog.  Then, when they want to log out, I need a page explaining to them how to do that and why they must do it this way.  Fortunately, my users are pretty smart and they understand; I haven't had any complaints about it.  But not everyone has that luxury.  And it would be really nice if my users didn't have to deal with that either.  But they do, because while HTTP authentication is more or less complete from a *technical standpoint*, it's incomplete from a *user experience* standpoint.  And that is what we have an opportunity to fix in HTML5.  Isn't one of the goals of the new standard to improve user experience for existing technologies?  And to improve security on the web?  Most people don't use HTTP auth because the user experience is inferior.

And yes, I use the credential forwarding system I spoke of before.  I may not have quite as many users as Wikipedia, but my database doesn't seem to have a problem with it (PostgreSQL).  I also don't have *nearly* as much hardware as Wikipedia has.

> explained a specific real-world attack that could be carried out on those
> applications, and explained how HTTP auth would have prevented that attack.  If
> the application is secret or in-house, you could give a basic rundown of how
> it's set up, as much as is necessary to understand the attack.
> 

So you want a real-world *attack* scenario?  OK.  I happen to have an installation of MediaWiki on a shared hosting environment.  I'm not going to publish where it's hosted or what the URL is, because it *is* vulnerable to attack.

My shared host (a very popular one) has PHP set up with the *default* session store, which is flat files in /tmp.  This directory is readable to anyone.  Usually, the flat files themselves are only readable to the user that created them.  But for the case where someone wants mod_php for their domain instead of FastCGI, the files must be created as the apache user.  Despite warnings about the security implications of this, mod_php is a popular choice, because it's the only simple way to get certain features like full header and environment information.  And this is not hypothetical; I can easily see that a lot of people on my shared box are, *in fact and in the real world* using mod_php.

Since I am on that shared server, I can easily execute arbitrary code as the apache user.  My MediaWiki has an admin account called WikiSysop, and let's say someone else who has shell access to that box (remember, it's a shared server) wants to login as WikiSysop and wreak havoc:

[mybox]$ grep WikiSysop /tmp/*
/tmp/sess_4i34n8ih9kr3afticmr4i6kp73:wsUserID|i:1;wsToken|s:32:"177cb3a9c9d0fbbcf624295d59329375";wsUserName|s:9:"WikiSysop";wsLoginToken|N;

Whoops.  Looks like WikiSysop is currently logged in, and I have all the info I need to steal his session.  I have the session ID (it's part of the filename: 4i34n8ih9kr3afticmr4i6kp73), I have the User ID, and the UserName.  These three values simply go into some cookies, and in a flash I am logged in as WikiSysop.  I just tested this attack in the real world.  And it worked.  That's about as real-world as I can give you, because I'm not going to go ahead and break into someone else's site.

> 
> With cookies, there's not much need to save passwords.  Lots of sites will let
> you stay logged in for months or forever without having to re-enter your
> password.  But if you use basic HTTP auth, then you either have to re-enter
> your password for every site every time you start the browser, or have it save
> the passwords to disk.  So it will encourage password saving much more
> strongly.
> 

I don't have any statistics on saved passwords.  But lots of people I know, including myself, use them heavily.  I protect mine with a master password, so I'm not that worried about it.  I'm not sure if other people do that.  But anecdotally, whether it's HTTP auth or cookie auth has little bearing on whether I save the password, even if there is a "keep me logged in for a while" option.  Eventually I will not be logged in anymore, so I save the password.  Most people don't even understand the distinction, so it will have even less bearing for them.

> 
> Yes, but a stolen cookie is much less damaging.  It can have a limited
> lifetime, can be canceled, and doesn't give information about the password
> itself (which is significant when users use the same or similar passwords
> across sites, which most do).  A leaked password is much worse.

I'm not sure I agree with this.  In the attack I executed above, I am now logged in as WikiSysop without entering his password.  I can do pretty much anything he can do, short of changing his password (since that requires the old password - good move).  But by the time that cookie expires or is "canceled" (not sure how you would figure out that you need to cancel it, "in the real world"), I can do a lot of damage since I am logged in as what basically amounts to the root user for that MediaWiki install.

I agree that a leaked password is far more damaging, but to use your own argument, you haven't provided a "real-world" situation where HTTP auth invariably leads to a leaked password.  You have just assumed that it's true.  If you don't want to use SSL on your site, you can use Digest auth instead of Basic auth.  You lose the credential forwarding ability, but you still remove the risk of attacks like the one I performed above.  And if you want to talk real-world, and you want to discount financial or other sites with sensitive data, then shared hosting is about as real as it gets.  

I know you're going to say that the shared host and the site operator are at fault for allowing that attack; the shared host should have done more to lock down those session files, and the site operator shouldn't have chosen mod_php despite the warnings of the hosting guys, and they shouldn't have been using flat file sessions in the first place (let alone in /tmp).  But that happens ALL THE TIME in the real world. A HUGE percentage of shared-host PHP sites are running with poor security against the other users of that box (and if the attacker isn't currently one of those other users, he can become one for a low monthly price!)

If the hosts leave any default settings for PHP or even for the OS in general (by the way, all the MediaWiki config files are by readable to apache, so I can also steal other peoples' database credentials, and they can steal mine.  In the real world.) then they are open for attack.  That could be mitigated by them fanatically locking down files, forbidding mod_php, using VPS instead of shared hosting, etc.  But it will be nigh on impossible for them to lock down every single avenue for people to steal each other's sessions or configs.  If HTTP authentication were fixed, and if database permissions were proper, there would *be a way* to prevent session hijacking with 100% certainty (by not using sessions) and prevent, or at least limit the damage from, database credential theft.  Currently, most shared hosting customers (probably a large percentage of your userbase for MediaWiki) simply assume that they're safe, and a lot of the time they're wrong.  Obviously, they shouldn't assume that -- but they do in the *real world*.  What if they weren't wrong and they actually *were* safe?  That seems like a worthwhile goal to me.  Do you agree? 

Somehow, I have a feeling that my descriptions aren't real-world enough; or that even though I've basically written a dissertation on this by now, you're going to say I wasn't specific enough about something.  But, as you've probably figured out already, I love the sound of my own voice and I'm happy to continue debating the details for as long as you can stand it ;)

Jeremy

(In reply to comment #9)
> You do?  I'm happy to keep up the debate; I'm just worried about the half-dozen
> people who get CC'ed every time we post a reply.

Phooey, they can unsubscribe themselves if they object.  We're talking about the merits of your proposed spec changes, so it's not like it's off-topic.  Technically.

> But they do, because while HTTP authentication is more or less
> complete from a *technical standpoint*, it's incomplete from a *user
> experience* standpoint.  And that is what we have an opportunity to fix in
> HTML5.  Isn't one of the goals of the new standard to improve user experience
> for existing technologies?  And to improve security on the web?  Most people
> don't use HTTP auth because the user experience is inferior.

Well, the question is a cost-benefit tradeoff for browser implementers.  Realistically this has no chance of happening because web developers almost exclusively use cookies, so implementers have no incentive to improve HTTP auth -- they have other things to spend their limited time on that their users are more concerned with.  I mean, if they cared about HTTP auth even slightly, they could start by making the UI for it not completely horrible, but evidently they don't.  So your proposal is certainly a non-starter, since it doesn't even have guaranteed returns (authors might not use it).

> And yes, I use the credential forwarding system I spoke of before.  I may not
> have quite as many users as Wikipedia, but my database doesn't seem to have a
> problem with it (PostgreSQL).  I also don't have *nearly* as much hardware as
> Wikipedia has.

Wikipedia has less hardware than you'd think.  Less than a thousand servers last I checked, only a few dozen database servers.  (Granted, probably more than you.)  Anyway, I have heard that PostgreSQL is designed for much more granular permissions than MySQL, so I wouldn't be surprised if something like this were doable using it.

> My shared host (a very popular one) has PHP set up with the *default* session
> store, which is flat files in /tmp.  This directory is readable to anyone. 
> Usually, the flat files themselves are only readable to the user that created
> them.  But for the case where someone wants mod_php for their domain instead of
> FastCGI, the files must be created as the apache user.  Despite warnings about
> the security implications of this, mod_php is a popular choice, because it's
> the only simple way to get certain features like full header and environment
> information.  And this is not hypothetical; I can easily see that a lot of
> people on my shared box are, *in fact and in the real world* using mod_php.

If you use FastCGI, they have to be created as the FastCGI user, so it's really no different.  The way to get around it is to use CGI, but shared hosts often don't do that because it's inefficient, since it has to fork() on each request.  (But some do use CGI, like 1&1.)  Most hosts that use mod_php also use open_basedir and friends, so you can't open other people's files anyway in practice in most cases, to be fair.

But yes, this is definitely a real-world scenario.  I'd say it's not an issue because if you can read the files in /tmp, you can read MediaWiki's configuration too, and so connect to the database directly.  But you'd counter that if MediaWiki used your auth forwarding scheme, it wouldn't have to store the database passwords anywhere.  Clever.  I guess there would be no obstacles if you move enough of the processing logic to the database servers.

I will point out that the attack scenario only works if the attacker has the right only to run PHP code, not arbitrary code.  Like if PHP is configured with exec() and friends disabled, but not open_basedir for some reason.  If you can run arbitrary code as the Apache user, you can attach a debugger to all the Apache children and grab the HTTP headers off the wire to get the passwords.  This is a bit harder than using a cookie from a plaintext file, but still easy, script-kiddie stuff -- strace on Linux should be enough if you can get it not to truncate the syscall arguments (I assume there's some option).

Okay, so it's possible in real life to design such a system.  Practically speaking, though, very few people do or will, even if HTTP auth worked better.  It's much easier to just keep off-site backups and assume that your database servers are compromised if your application servers are.  Elaborate privilege separation tends to only be found in a minority of applications in any field (Chrome, postfix, . . .) because it takes a lot of effort.  Unfortunately for you, web standards only try to cover the most common use-cases, so you're out of luck.  (That's a prediction, not an assurance, but I'd bet you ten bucks.)

> I'm not sure I agree with this.  In the attack I executed above, I am now
> logged in as WikiSysop without entering his password.  I can do pretty much
> anything he can do, short of changing his password (since that requires the old
> password - good move).  But by the time that cookie expires or is "canceled"
> (not sure how you would figure out that you need to cancel it, "in the real
> world"), I can do a lot of damage since I am logged in as what basically
> amounts to the root user for that MediaWiki install.

Actually, as it happens, MediaWiki users can't really do any damage even if they have full privileges.  Even privileged operations are all logged and reversible.  (If you're sufficiently creative you can make them a real pain in the neck to reverse, granted.)  The only way to do real damage is to edit LocalSettings.php or directly access the database.  But that's a side point.  :)

By default, MediaWiki expires cookies after thirty days.  So you have that long at most.  This still allows a short-term vandalism spree, but stops you from doing any kind of subtle long-term attacks, like snooping indefinitely on a private wiki.  So that's a security win, even if there's no explicit way to expire cookies prematurely (not sure offhand).

> I agree that a leaked password is far more damaging, but to use your own
> argument, you haven't provided a "real-world" situation where HTTP auth
> invariably leads to a leaked password.

How about: <http://codebutler.com/firesheep>.  Anyone on an unsecured wireless connection to a non-HTTPS website will leak all their HTTP requests in full.  Since sites generally use cookies, Firesheep can only reliably hijack current sessions, not permanently take over accounts.  If you had a specific site you were targeting, you might be able to launch an active attack that inserted malicious JavaScript to lure the user into entering their password, but that's much harder to pull off and easier to detect than just snooping on HTTP headers, and it requires user interaction no matter what.

Of course, this doesn't happen if you use HTTPS.  But there are lots of practical reasons why that's a pain, both logistic ("oops, I mistyped a line and now all my users are getting scary cert errors") and performance (extra RTs, much more expensive for caching proxies).  Hopefully DNSSEC will make this easier, but I'm not going to hold my breath on most sites switching to HTTPS anytime soon.

> I know you're going to say that the shared host and the site operator are at
> fault for allowing that attack

Of course not, that's silly.  The goal is to prevent attacks, not to say we don't have to worry about particular classes of attacks because they're someone else's fault.  As far as possible, web standards are designed to remain functional and secure even in the hands of authors and site admins whose idea of a development process is to copy-paste code you found on Google and tweak it until it seems to do what you want.

> Somehow, I have a feeling that my descriptions aren't real-world enough; or
> that even though I've basically written a dissertation on this by now, you're
> going to say I wasn't specific enough about something.

Nope, I'm happy to concede at this point that what you describe is *possible* in practice.  But it's just not practical for the large majority of websites.  The security benefit is greatly outweighed by the added development costs.  Certainly for most websites, where the worst case is that a bunch of pimply teenagers have to temporarily switch to Facebook to coordinate their WoW raids until the host has reinstalled the software for them.  I suspect it's actually not economical for any website, but I can't make an informed claim on that score, since I'm not involved in high-stakes websites.  (Downtime is when Wikipedia gets the most donations!)

> But, as you've probably
> figured out already, I love the sound of my own voice and I'm happy to continue
> debating the details for as long as you can stand it ;)

Glad to see I'm not alone!  I should have spent the last 45 minutes or so working on the DOM Range spec for Google instead of arguing with you.  But hopefully now we agree, although our opinions are completely irrelevant because this is going to be WONTFIX with boilerplate "get implementers to express interest first" anyway.

EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Rejected
Change Description: no spec change
Rationale: 

I disagree with the premise of this proposal:

> HTTP Authentication is important, because it is the only way
> to execute a request with 100% certainty that the user has provided an
> authentication secret.

A form asking the same question as HTTP auth is no less reliable, and HTTP auth does not guarantee that the user provided anything, rather than the UA providing it.

HTTP Authentication has IMHO not been demonstrated as being important for the interactive Web use case. (It has some more compelling uses in automated environments, but those are out of scope here.)


My recommendation would be to follow one or more of the following paths:

* If you have very specific problems with HTML, file bugs specifically for those problems, including clear rationale for why we need to address those problems.

* If you have specific proposals, approach browser vendors to get them to implement them; if they do, and if they are found to be good, then we would add them to the spec.

* Escalate this bug as described at the top of this comment, and submit a detailed changed proposal to the HTMLWG to convince the WG chairs that this proposal is worth adopting, and then convince browser vendors to implement it.

Incidentally, zewt on #whatwg pointed out you can get all the security benefits of your scheme using cookies:

<zewt> (AryehGregor: not necessarily useful to that person, but if I really needed that, I'd probably do something along the lines of storing an encrypted password in the cookie with a key on the server, so the server can decrypt it for each request and then throw it away)

That way the server still doesn't have to store any credential info that an attacker could profitably compromise.

(In reply to comment #12)
> Incidentally, zewt on #whatwg pointed out you can get all the security benefits
> of your scheme using cookies:
> 
> <zewt> (AryehGregor: not necessarily useful to that person, but if I really
> needed that, I'd probably do something along the lines of storing an encrypted
> password in the cookie with a key on the server, so the server can decrypt it
> for each request and then throw it away)
> 
> That way the server still doesn't have to store any credential info that an
> attacker could profitably compromise.

1. For the spoofing attack, now I just have to figure out where you store the key.  Or are you generating a new key on each request?  No matter how many levels of indirection you put between me and the stored session key, I can still get to it.  There is simply no way to make it impossible; only really hard.

2. You're still storing your database credentials using this mechanism.

Anyway, Hixie has made the decision.  If he doesn't see what I'm trying to say then I doubt anyone else on the WG would.  What a shame - we had a chance here to provide a way to pass credentials directly from the user's brain to backend services without having to store them on the web server.  Would have been great for security; in fact I think it would have soon become an industry-wide best practice if the UA support was there.  Oh well.  I'll set it to CLOSED.

(In reply to comment #13)
> 1. For the spoofing attack, now I just have to figure out where you store the
> key.  Or are you generating a new key on each request?  No matter how many
> levels of indirection you put between me and the stored session key, I can
> still get to it.  There is simply no way to make it impossible; only really
> hard.

No, you just use the same key.  If the attacker gets the key, so what?  They still don't have anything useful until they get a cookie, and the server doesn't store the cookies.  If they get the key plus also the contents of an HTTP request, they get that specific user's password, but that problem occurs in your scheme too.

> 2. You're still storing your database credentials using this mechanism.

No, you're not.  The application server only stores the key, which is useless without the cookie.  It throws away the cookie as soon as it gets it and stores no info related to it, just like with username/password in HTTP auth.  The cookie here isn't used as a key into some kind of session info storage, it contains the actual login info in encrypted form.

Actually, for that matter, you could dispense with the encryption.  Just have the cookie contain the username and password in plaintext.  This is precisely equivalent in security to HTTP auth on a protocol level, because precisely the same information is included in every request.  With the username and password included in the request, the web server needs to store no auth info at all.

(In reply to comment #14)
> 
> > 2. You're still storing your database credentials using this mechanism.
> 
> No, you're not.

Ah, I completely misunderstood what you were suggesting.  And it's a pretty great idea.  It had never occurred to me that you can use cookies in pretty much the same way as auth credentials as long as you use them *directly*.  I am so used to session stores that I completely forgot that cookies can store more than just a session ID - and you can completely control them with a wide variety of approaches.

Thanks for the idea!  And my thanks to Zewt as well.  I still think from a theoretical standpoint that improving HTTP Auth would be a better choice.  But from a practical standpoint, you're right - it seems like cookies could be used in much the same way - you just have to be more careful with them (i.e. encryption).

mass-move component to LC1