How to Take Care of an Overheating Power Adapter

How to Take Care of an Overheating Power Adapter

 

What to do, what to do?

I love my laptop. Sometimes I think I use it more than I should, such as when try to move my charger cable only to find that my adapter is piping hot!

Why do adapters get hot? Well, they have to convert the Alternating Current (vibrating electrons) from your wall socket into the Direct Current (moving electrons) that electronic components need to function. Basically, it converts one form of energy into another, just as a car engine converts fossil fuel into mechanical push, and just like a car engine, the process isn’t 100% efficient and some (or a lot) of the source energy is lost as heat during the process. This is generally why old, beat up electronics might consume more energy than when they where new.

What to do? Well, in all honesty, if the adapter is getting too hot, chances are you should replace it.

What if I can’t or don’t want to buy a new adapter? If possible, you should firstly try to see if the adapter is still producing the voltage it should, if it’s too high or too low it might cause issues. For that you need a voltmeter, if you have a voltmeter you probably already know how to proceed with that… if you don’t have a voltmeter you’re probably better off buying a new adapter, they should cost about the same.

Anyways, once you’ve decided to NOT buy a new adapter (and dismissed plan A):

Make sure it is well ventilated. The plastic needs air to flow around it in order to not overheat. Though it could work while being super-hot, it could be super-ruined once it gets hot enough to damage the materials in it. Simply putting it on a hard surface and AWAY from any cloth should be enough. Also keep it away from you or anything else that it warmer than room-temperature (this includes pets, which could also chew on cables)

Once it’s well ventilated there shouldn’t be much issues. It really isn’t a big deal. A common error is trying to disconnect a laptop charger in order to give the charger some time to “cool”, this actually does the opposite, since it’ll just get hotter than before once you connect it again and your laptop (or other electronics) sucks up its operating energy PLUS charges the battery at the same time. The temperature won’t just go up and up if you leave it in, this is because the hotter it gets, the easier it is for air to absorb the heat energy, which will lead to a point of balance that depends on a bunch of stuff that’s probably enough to fill another article. NOTE: I;d especially want to keep it away from anything flammable, just in case it gets TOO hot.

Expert mode: attach a heat sink to it. This will make it easier for the air to absorb the lost energy (heat) and keep the temperature down. A quick search on Amazon or Ebay should bring up some results. A few small VGA heat sinks should do the trick. Using a larger heat sink would mean finding out which face of the adapter hold the inner heat sink which might mean opening it up (or just put it on the hotter face) (NOTE: using a larger heat sink might require sanding one of the surfaces, if you don’t already know the risks involved in doing this, don’t even try it… but you’re probably too cool to pay attention to me, right?).

Anyways, if you have any comments or questions just leave them here, I’d be more than glad to polish up this article a bit in order to cover your specific concerns (because you are so important to me).

Good luck!

DDD (DOMAIN DRIVEN DESIGN).

shared from : http://www.codeproject.com/Articles/339725/Domain-Driven-Design-Clear-Your-Concepts-Before-Yo

 

Start Developing a New Application 

What we traditionally do when we start a business application? We read the spec and find the functionalities. We break down tasks. In most of the cases the goal of the breakdown is to come up with an estimation and plan of works. We do the estimation. We distribute the works among team members. We design the database schema – sometimes by the team leader or sometimes by the respective developer. We start coding.

So?  What’s wrong with this approach? We have been doing good! Don’t we? 
The answer is YES and NO! Yes we are doing good in delivering our projects. But NO! We are not doing good in maintaining and extending our projects!

Think about all of the projects you have worked last few years in the traditional approach. Did you ever face any of the issues below?

  1. Your project has the same functionality implemented in the same way or different in different places.
  2. You have more than one object for the same item.
  3. You have objects that have properties that are not actually attributes of that object.
  4. You have no or very poor relationship among related items.
  5. Looking at your objects it is not possible to understand what actually the whole application is all about.

I am sure you have been facing these issues regularly. But do you know why?  The reason is, traditional approach do not guide us designing the system in Up to Bottom fashion. Rather it tempted us to design the system in Bottom-Up fashion. See, when you design a system you need to know what as a whole the application will do? What is the goal the client is trying to achieve? Then, from the top level goal you come up with different smaller functionalities that will eventually allow the users to achieve the top level goal. 

 

But when you design in bottom-up approach, you first design for the granular functionalities, and you have little or no knowledge how this functionality will be used from the Top level and how the Top level functionalities will actually look like.

Have you ever heard that a developer of your team is talking like he does not have the domain knowledge of the whole application? Perhaps yes! I think you can understand the reason. Cause, the design of the application does not represent the domain of the system. And so, developers know only the portions they worked. This is Sad! Isn’t it? 

So, is traditional approach – “Designing the application starting from database” a throw away concept?  Not really! But if you have a complex application to develop, this bottom-up design approach does not dictate you to come up with a proper object oriented design. 

What is the solution then? 

The solution is DDD (DOMAIN DRIVEN DESIGN).

 

 

What is DDD?

Domain-driven design is not a technology or a methodology. DDD provides a structure of practices and terminology for making design decisions that focus and accelerate software projects dealing with complicated domains.

– Wikipedia

 

 

Concepts to cover in this article:

  1. Understanding the Domain.
  2. Ubiquitous Language.
  3. Contexts and Bounded Contexts.
  4. Entities and Value Objects.
  5. Aggregates and Aggregate Roots.
  6. Persistence Ignorance. 
  7. Repository. 
  8. Domain Service. 

 

In this article I will try to avoid becoming too technical, rather I will try to go through different concepts of DDD being close to the real world.  I will try not to show any code here. Because I believe if you understand the concept and starts thinking in DDD way, implementation is easy. The toughest part is to tune your thinking process!

 

Understanding the Domain

A sphere of knowledge, influence, or activity. The subject area to which the user applies a program is the domain of the software.
– Wikipedia

 

Do you get a feeling what is domain from this definition?  Can you tell what is the domain of the project you are working on at this moment? Can you tell what is the domain of the famous website YouTube?

In this article I would like to go through a real world example to give you the feeling how to start analyzing your project driven by your domain. This example may not be related with application development but as the goal is to tune our thinking top to bottom manner, it will be useful. But again, we will go through the technical terms of DDD too! 

Let’s say you are engaged to design a building. The requirement is:

  • You have a defined amount of land
  • Your building will have 6 floors.
  • Each floor will have 4 apartments.

 

What is your domain here?

The domain is Building(?). It could be. But note that, if you consider Building as your domain you may miss few granular details for your requirement. The building you are going to design must have design for apartments where people will live. So, a general term “Building” can make us miss few details. So, we may narrow down our domain to “Residential Building”.

Now, when you talk about your work with engineers and also with the people who engaged you to design the building, the term “Residential Building” is more meaningful for everybody concerned. Did you mark very small change in language here? The contractor is telling you to design a building where there will be 4 apartments in each of the 6 floors. Now, if you send an engineer to the site telling him we will need to construct a building there, they might not consider many attributes that a residential building must have. On the other hand if you use the term “Residential Building”, most likely he will come with a valid analysis.
This is how we come to an “Ubiquitous Language”.

 

Ubiquitous Language

The concept is simple, that developers and the business should share a common language that both understand to mean the same things, and more importantly, that is set in business terminology, not technical terminology.

 

 

More Example of Ubiquitous Language:

Example 1:

Wrong Language:

The length and width ratio of the smaller bed rooms would be 4:3.

 

 

Correct Language:

The children’s bed room’s length will be 20 ft and width will be 15 ft.
Note that, to the owner of the building “smaller room”, “ratio” – all these things could be very technical terms. Rather it is easier for him to understand children’s room, guest room, living room etc. And explicit measurement is more meaningful to him.

 

 

Example 2:

Let us see an example from software perspective.

 

 

Wrong language:

In search functionality we will consider inflectional and thesaurus feature of sql server to make the search more relevant. Additionally we will also exclude the stop words from the search to make it more accurate. 
Note that, your domain expert may not be a technical person and thus he may not understand what you meant by the words “Inflectional”, “Thesaurus”, “Stop word” etc.

Correct language:

In the search functionality we will consider all the synonyms of the search phrase so that it does not exclude relevant results. Moreover we will not differentiate any search word by its number (singular or plural), tense, participle etc so that the result becomes more accurate. Additionally as expected in any search, we will ignore all the noise words that do not have any value in the search. Such noise words could be “am”, “but”, “where”, “about” etc.
Do you see the difference in language here? Really a correct language can make all the involved parties think and understand in the same way.

 

Let’s come back to our “Residential Building” domain. Look, you can proceed with the residential building design as a single task and address the whole thing together. But will it really be very wise way to do? Note that, if you just consider this a single unit of work you may miss many things. Designing a building is related to so many things. For example: you need to consider ventilation, utility, parking space, community space etc.

Now you see, different other contexts are coming up. This is how the concept “Context” and “Bounded Context” comes up in Domain Driven Design.

 

Contexts and Bounded Contexts

A Bounded Context can be considered as a miniature application, containing itss own Domain, own code and persistence mechanisms. Within a Bounded Context, there should be logical consistency; each Bounded Context should be independent of any other Bounded Context.
More Example of Bounded Context:
Think of an e-Commerce system. Initially you can tell it is an application of shopping context. But if you look more closely, you will see there are other contexts too. Like: Inventory, Delivery, Accounts etc.

 

Dividing a large application among different bounded contexts properly will allow you to make your application more modular, will help you to separate different concerns and will make the application easy to manage and enhance. Each of these Bounded Contexts has a specific responsibility, and can operate in a semiautonomous fashion. By splitting these apart it becomes more obvious to find where logic should sit, and you can avoid that BBOM (Big ball of mud) J

What is BBOM? 

A Big Ball of Mud is a haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated. The overall structure of the system may never have been well defined. 

– Brian Foote and Joseph Yoder, Big Ball of Mud. Fourth Conference on Patterns Languages of Programs (PLoP ’97/EuroPLoP ’97) Monticello, Illinois, September 1997

 

Our all time objective should be to avoid BBOM

Again with the “Residential Building Domain”. So, we could have several bounded contexts:

 

  • Electricity supply
  • Car parking
  • Apartment 
  • Etc.

Let’s talk about the apartment. The apartment is basically a combination of different rooms. The rooms have different elements inside like windows, doors etc. Now I have 2 questions to you about the windows of the room.

 

Question1: Can you imagine a window without a room?

Question2: Does a window have any identity without the room it is residing in?

Answering these questions will expose the following concepts of DDD.

 

  1. Entity.
  2. Value Object.
  3. Aggregates & Aggregate root.

 

 

Entity

“This is my Entity, there are many like it, but this one is mine.” 

 

The key defining characteristic of an Entity is that it has an Identity – it is unique within the system, and no other Entity, no matter how similar is, the same Entity unless it has the same Identity.

 

Examples: 

  1. Your bed room in the apartment.
  2. Contract in Facebook.
  3. Article in CodeProject.

 

 

Value Object 

The key defining characteristic of a Value Object is it has no Identity. Ok, perhaps a little simplistic, but the intention of a Value Object is to represent something by its attributes only. Two value objects may have identical attributes, in which case they are identical. They don’t however have any value other than by virtue of their attributes. Another aspect common to value objects is that they should probably be immutable, once created they cannot be changed or altered. You can create a new one, and as they have no identity, that is just the same as changing another one.

 

 

Example:

  1. Windows in the rooms
  2. Address of any person in your website.
  3. SearchCriteria of your search.

 

Note:  A value object can become an entity depending on the situation. Can you find a scenario like that? If the requirement of the search functionality of your application says that, the search criteria should be saved in the database and the user can do the same search from the list of saved search criteria’s. In this scenario SearchCriteria has its own identity and thus it is an entity instead of being a value object. 

 

Now you know what entity is and what value object in DDD is. In domain driven design entities and value objects can exist independently. But in some cases the relation can be such that, an entity or VO has no value without its context.

 

Example: 

  1. A window can only be defined if there is a room. 
  2. An order note can only exist if an order is placed.
  3. A question detail can only be there if a question is asked.

 

Very simple is not it? Believe me, now you know what Aggregate and Aggregate root is in DDD.

Aggregate and Aggregate Root

In the examples given above – 

 

  • Room, Order and Question are our aggregate roots. 
  • On the other hand window, order note and question detail are our aggregates. 

“A cluster of associated objects that are treated as a unit with regard to data changes.”
All objects of the clusters should be treated as aggregate.
All external access to the cluster is through a single root Entity. This root entity is defined as aggregate root.

 

Example:

  1. A question detail should no way be saved unless the corresponding question is saved.
  2. A question detail should no way be retrieved unless the  corosponding question is retrieved. 

 

Here Question is the Aggregate root and Question Detail is the aggregate. Aggregates and Aggregates Root are very important concepts of DDD.

So far we have talked about domain, objects/entities, contexts, aggregates etc. What about the Database? Is that something we have missed? Isn’t it something should come in the design?

The answer is NO!  DDD is a persistence ignorant approach.

 

Persistence Ignorance

In domain driven design your objective is to create a model of the domain. You need to identify what are the items (objects) you need to accomplish the desired functionalities of your application. You need to identify the relationships among different objects and how they interact among themselves. You need to find if the business goal of your client is achievable using your domain model. Where is the existence of database here?  You do not need to know how and where the data of your domain will persist or even if the data do need to persist while you do the model of the domain.

 

This ignorance about your persistence medium will make your domain model free from any coupling with the persistence layer of the application. This will eventually separate the concerns of the persistence and its communication mechanism from your domain model. In result your application will be free from coupling with any data store and will be very easily unit testable.
But Yes! In a real application you do need to have a database. But your domain model will have no knowledge about that. All it will know is the “Repository” which will eventually manage your application’s persistence concern.

 

Repository

Can you tell me what the meaning of the English word “Repository” is? 

 

Repository commonly refers to a location for storage, often for safety or preservation.
– Wikipedia

As I have already said your domain model will not know any database. What it will know is, there is a repository in the system and that repository will be responsible to store your data and to retrieve your data. It is no way a concern of your domain model how and where data will persist. So, it can be Sql server, oracle, xml, text file or anything else. I hope now you got a sense what a repository means in DDD.

Let’s become little more technical.

Repository Mediates between the domain and data mapping using a collection-like interface for accessing domain objects. It is more like a facade to your data store that pretend like a collection of your domain.

Repository Is Not A Data Access Layer. 

Note that repository doesn’t talk in terms of “data”, it talks in terms of Aggregate Roots. You can tell your repository to add an Aggregate Root into its collection, or you can ask it for a particular Aggregate Root. When you remember that Aggregate Roots may comprise one or many Entities and Value Objects, this makes it fairly different to a traditional DAL that returns you back a set of rows from your database tables. 

 

Implementation Strategy of Repository:

As I said, Repository is a design pattern that is used in DDD to handle the persistence concern. The detail of this pattern is out of the scope of this article. However, here I am trying tell in minimum how we may achieve a repository implementation. 

 

 

  1. 1st of all you will have an interface – IRepository that should be generic.
  2. You will have an abstract implementation of the IRepository interface.
  3. You will have interface INhRepository  for your persistence mechanism (i.e. Nhibernate) this will inherit from IReposiroty
  4. You will have implementation of INhReposiroty in a class like “NhReposirory”.
  5. Finally you may have a generic implementation of the repository that will have default implementations of all the common methods of the repository. 
  6. Like NHGenericRepository that inherits from NhRepository and implments IGenericNhReposirtory.
  7. You will specific repository for you Aggregate Roots, that will be extended from NHGenericRepository.
  8. Your application will use service locator to find which repository the application will use. 

 

 

Domain Service

Domain service is another important concept of DDD. If Entities and Value Objects are the “things” in your domain, the services are a way of dealing with actions, operations and activities.

 

Shouldn’t Logic Be on the Entities Directly?

Yes, it really should. We should be modeling our Entities with the logic that relates to them and their children. But, there are occasions when we need to deal with complex operations or external responsibilities or maybe we need to expose the actions of the aggregate roots to the external world. This is why creating a domain service for different aggregate root is a good idea. You can consider the domain services as façade layer of the business logics and operations of your domain.  

End Words 

In this article I have tried to introduce the basic concepts and terminologies of Domain Driven Design with examples of real world. The goal was to make you feel comfortable with DDD world. But really developing applications with DDD is a big challenge. The more you love and practice DDD concepts while you design your object model, the more accuracy you will gain in your design. As I said before the most important thing is, you must think in Domain Driven Way. If you don’t you will hugely suffer when your application is a real complex one.

Great DDD Resources 

http://www.dddcommunity.org/  

http://thinkddd.com/ 

http://www.infoq.com/minibooks/domain-driven-design-quickly 

http://thinkddd.com/assets/2/Domain_Driven_Design_-_Step_by_Step.pdf 

http://www.dddcommunity.org/library/young_2010 

http://www.dddcommunity.org/library/evans_2010 

http://blog.fossmo.net/post/Command-and-Query-Responsibility-Segregation-(CQRS).aspx 

http://msdn.microsoft.com/en-us/library/ff649690.aspx 

 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

study->infosec_interview_questions

http://www.danielmiessler.com/study/infosec_interview_questions/

 

[ Check out my latest post on the HP Security Blog: “The Secure Web Series, Part 3: Protecting Against Cross-site Request Forgery (CSRF)” ]

What follows is a list of questions for use in vetting candidates for positions in Information Security. Many of the questions are designed to get the candidate to think, and to articulate that thought process in a scenario where preparation was not possible. Observing these types of responses is often as important as the actual answers.

I’ve mixed technical questions with those that are more theory and opinion-based, and they are also mixed in terms of difficulty. They are also generally separated into categories, and a number of trick questions are included. The goal of such questions is to expose glaring technical weakness that will manifest later in the workplace, not to be cute. I also include with each question a few words on expected/common responses.

General

Are open-source projects more or less secure than proprietary ones?

The answer to this question is often very telling about a given candidate. It shows 1) whether or not they know what they’re talking about in terms of development, and 2) it really illustrates the maturity of the individual (a common theme among my questions). My main goal here is to get them to show me pros and cons for each. If I just get the “many eyes” regurgitation then I’ll know he’s read Slashdot and not much else. And if I just get the “people in China can put anything in the kernel” routine then I’ll know he’s not so good at looking at the complete picture.

The ideal answer involves the size of the project, how many developers are working on it (and what their backgrounds are), and most importantly — quality control. In short, there’s no way to tell the quality of a project simply by knowing that it’s either open-source or proprietary. There are many examples of horribly insecure applications that came from both camps.

How do you change your DNS settings in Linux/Windows?

Here you’re looking for a quick comeback for any position that will involve system administration (see system security). If they don’t know how to change their DNS server in the two most popular operating systems in the world, then you’re likely working with someone very junior or otherwise highly abstracted from the real world.

What’s the difference between encoding, encryption, and hashing?

Encoding is designed to protect the integrity of data as it crosses networks and systems, i.e. to keep its original message upon arriving, and it isn’t primarily a security function. It is easily reversible because the system for encoding is almost necessarily and by definition in wide use. Encryption is designed purely for confidentiality and is reversible only if you have the appropriate key/keys. With hashing the operation is one-way (non-reversible), and the output is of a fixed length that is usually much smaller than the input.

Who do you look up to within the field of Information Security? Why?

A standard question type. All we’re looking for here is to see if they pay attention to the industry leaders, and to possibly glean some more insight into how they approach security. If they name a bunch of hackers/criminals that’ll tell you one thing, and if they name a few of the pioneers that’ll say another. If they don’t know anyone in Security, we’ll consider closely what position you’re hiring them for. Hopefully it isn’t a junior position.

Where do you get your security news from?

Here I’m looking to see how in tune they are with the security community. Answers I’m looking for include things like Team Cymru, Reddit, Twitter, etc. The exact sources don’t really matter. What does matter is that he doesn’t respond with, “I go to the CNET website.”, or, “I wait until someone tells me about events.”. It’s these types of answers that will tell you he’s likely not on top of things.

If you had to both encrypt and compress data during transmission, which would you do first, and why?

If they don’t know the answer immediately it’s ok. The key is how they react. Do they panic, or do they enjoy the challenge and think through it? I was asked this question during an interview at Cisco. I told the interviewer that I didn’t know the answer but that I needed just a few seconds to figure it out. I thought out loud and within 10 seconds gave him my answer: “Compress then encrypt. If you encrypt first you’ll have nothing but random data to work with, which will destroy any potential benefit from compression.

What’s the difference between symmetric and public-key cryptography

Standard stuff here: single key vs. two keys, etc, etc.

In public-key cryptography you have a public and a private key, and you often perform both encryption and signing functions. Which key is used for which function?

You encrypt with the other person’s public key, and you sign with your own private. If they confuse the two, don’t put them in charge of your PKI project.

What kind of network do you have at home?

Good answers here are anything that shows you he’s a computer/technology/security enthusiast and not just someone looking for a paycheck. So if he’s got multiple systems running multiple operating systems you’re probably in good shape. What you don’t want to hear is, “I get enough computers when I’m at work…” I’ve yet to meet a serious security guy who doesn’t have a considerable home network–or at least access to one, even if it’s not at home.

Network Security

What port does ping work over?

A trick question, to be sure, but an important one. If he starts throwing out port numbers you may want to immediately move to the next candidate. Hint: ICMP is a layer 3 protocol (it doesn’t work over a port) A good variation of this question is to ask whether ping uses TCP or UDP. An answer of either is a fail, as those are layer 4 protocols.

How exactly does traceroute/tracert work at the protocol level?

This is a fairly technical question but it’s an important concept to understand. It’s not natively a “security” question really, but it shows you whether or not they like to understand how things work, which is crucial for an Infosec professional. If they get it right you can lighten up and offer extra credit for the difference between Linux and Windows versions.

The key point people usually miss is that each packet that’s sent out doesn’t go to a different place. Many people think that it first sends a packet to the first hop, gets a time. Then it sends a packet to the second hop, gets a time, and keeps going until it gets done. That’s incorrect. It actually keeps sending packets to the final destination; the only change is the TTL that’s used. The extra credit is the fact that Windows uses ICMP by default while Linux uses UDP.

What are Linux’s strengths and weaknesses vs. Windows?

Look for biases. Does he absolutely hate Windows and refuse to work with it? This is a sign of an immature hobbyist who will cause you problems in the future. Is he a Windows fanboy who hates Linux with a passion? If so just thank him for his time and show him out. Linux is everywhere in the security world.

Cryptographically speaking, what is the main method of building a shared secret over a public medium?

Diffie-Hellman. And if they get that right you can follow-up with the next one.

What’s the difference between Diffie-Hellman and RSA?

Diffie-Hellman is a key-exchange protocol, and RSA is an encryption/signing protocol. If they get that far, make sure they can elaborate on the actual difference, which is that one requires you to have key material beforehand (RSA), while the other does not (DH). Blank stares are undesirable.

What kind of attack is a standard Diffie-Hellman exchange vulnerable to?

Man-in-the-middle, as neither side is authenticated.

Application Security

Describe the last program or script that you wrote. What problem did it solve?

All we want to see here is if the color drains from the guy’s face. If he panics then we not only know he’s not a programmer (not necessarily bad), but that he’s afraid of programming (bad). I know it’s controversial, but I think that any high-level security guy needs some programming skills. They don’t need to be a God at it, but they need to understand the concepts and at least be able to muddle through some scripting when required.

How would you implement a secure login field on a high traffic website where performance is a consideration?

We’re looking for a basic understanding of the issue of wanting to serve the front page in HTTP, while needing to present the login form via HTTPs, and how they’d recommend doing that. A key piece of the answer should center around avoidance of the MiTM threat posed by pure HTTP. Blank stares here mean that they’ve never seen or heard of this problem, which means they’re not likely to be anything near pro level.

What is Cross-Site Request Forgery?

Not knowing this is more forgivable than not knowing what XSS is, but only for junior positions. Desired answer: when an attacker gets a victim’s browser to make requests, ideally with their credentials included, without their knowing. A solid example of this is when an IMG tag points to a URL associated with an action, e.g. http://foo.com/logout/. A victim just loading that page could potentially get logged out from foo.com, and their browser would have made the action, not them (since browsers load all IMG tags automatically).

How does one defend against CSRF?

Nonces required by the server for each page or each request is an accepted, albeit not foolproof, method. Again, we’re looking for recognition and basic understanding here–not a full, expert level dissertation on the subject. Adjust expectations according to the position you’re hiring for.

If you were a site administrator looking for incoming CSRF attacks, what would you look for?

This is a fun one, as it requires them to set some ground rules. Desired answers are things like, “Did we already implement nonces?”, or, “That depends on whether we already have controls in place…” Undesired answers are things like checking referrer headers, or wild panic.

What’s the difference between HTTP and HTML?

Obviously the answer is that one is the networking/application protocol and the other is the markup language, but again, the main thing you’re looking for is for him not to panic.

How does HTTP handle state?

It doesn’t, of course. Not natively. Good answers are things like “cookies”, but the best answer is that cookies are a hack to make up for the fact that HTTP doesn’t do it itself.

What exactly is Cross Site Scripting?

You’d be amazed at how many security people don’t know even the basics of this immensely important topic. We’re looking for them to say anything regarding an attacker getting a victim to run script content (usually JavaScript) within their browser.

What’s the difference between stored and reflected XSS?

Stored is on a static page or pulled from a database and displayed to the user directly. Reflected comes from the user in the form of a request (usually constructed by an attacker), and then gets run in the victim’s browser when the results are returned from the site.

What are the common defenses against XSS?

Input Validation/Output Sanitization, with focus on the latter.

Corporate/Risk

What’s the goal of information security within an organization?

This is a big one. What I look for is one of two approaches; the first is the über-lockdown approach, i.e. “To control access to information as much as possible, sir!” While admirable, this again shows a bit of immaturity. Not really in a bad way, just not quite what I’m looking for. A much better answer in my view is something along the lines of, “To help the organization succeed. ”This type of response shows that the individual understands that business is there to make money, and that we are there to help them do that. It is this sort of perspective that I think represents the highest level of security understanding—-a realization that security is there for the company and not the other way around.

What’s the difference between a threat, vulnerability, and a risk?

As weak as the CISSP is as a security certification it does teach some good concepts. Knowing basics like risk, vulnerability, threat, exposure, etc. (and being able to differentiate them) is important for a security professional. Ask as many of these as you’d like, but keep in mind that there are a few differing schools on this. Just look for solid answers that are self-consistent.

If you were to start a job as head engineer or CSO at a Fortune 500 company due to the previous guy being fired for incompetence, what would your priorities be? [Imagine you start on day one with no knowledge of the environment]

We don’t need a list here; we’re looking for the basics. Where is the important data? Who interacts with it? Network diagrams. Visibility touch points. Ingress and egress filtering. Previous vulnerability assessments. What’s being logged an audited? Etc. The key is to see that they could quickly prioritize, in just a few seconds, what would be the most important things to learn in an unknown situation.

As a corporate Information Security professional, what’s more important to focus on: threats or vulnerabilities?

This one is opinion-based, and we all have opinions. Focus on the quality of the argument put forth rather than whether or not they they chose the same as you, necessarily. My answer to this is that vulnerabilities should usually be the main focus since we in the corporate world usually have little control over the threats.

Another way to take that, however, is to say that the threats (in terms of vectors) will always remain the same, and that the vulnerabilities we are fixing are only the known ones. Therefore we should be applying defense-in-depth based on threat modeling in addition to just keeping ourselves up to date.

Both are true, of course; the key is to hear what they have to say on the matter.

Advanced

If I’m on my laptop, here inside my company, and I have just plugged in my network cable. How many packets must leave my NIC in order to complete a traceroute to twitter.com?

The key here is that they need to factor in all layers: Ethernet, IP, DNS, ICMP/UDP, etc. And they need to consider round-trip times. What you’re looking for is a realization that this is the way to approach it, and an attempt to knock it out. A bad answer is the look of WTF on the fact of the interviewee.

How would you build the ultimate botnet?

Answers here can vary widely; you want to see them cover the basics: encryption, DNS rotation, the use of common protocols, obscuring the heartbeat, the mechanism for providing updates, etc. Again, poor answers are things like, “I don’t make them; I stop them.”

Scenario Role-Play

For special situations you may want to do the ultimate interview question type. This is a role-played scenario, where the candidate is a consultant and you control the environment. I had one of these during an interview and it was quite valuable.

So you tell them, for example, that they’ve been called in to help a client who’s received a call from their ISP stating that one or more computers on their network have been compromised. And it’s their job to fix it. They are now at the client site and are free to talk to you as the client (interviewing them), or to ask you as the controller of the environment, e.g. “I sniff the external connection using tcpdump on port 80. Do I see any connections to IP 8.8.8.8.” And you can then say yes or no, etc.

From there they continue to troubleshooting/investigating until they solve the problem or you discontinue the exercise due to frustration or pity.

Conclusion

Keep in mind that these are just filter questions, meaning that those who can’t answer them should possibly be pruned from consideration. But it does not mean that those who get them right should definitely be hired. For more on that, check out my /study/hiring/ post.

Feel free to contact me if you have any comments on the questions, or if you have an ideas for additions.

Recommended

IP, TCP, and HTTP

source : http://www.objc.io/issue-10/ip-tcp-http.html

IP, TCP, and HTTP

Issue #10 Syncing Data, March 2014

By Daniel Eggert

When an app communicates with a server, more often than not, that communication happens over HTTP. HTTP was developed for web browsers: when you enter http://www.objc.io into your browser, the browser talks to the server named http://www.objc.io using HTTP.

HTTP is an application protocol running at the application layer. There are several protocols layered on top of each other. The stack of layers is often depicted like this:

Application Layer -- e.g. HTTP
----
Transport Layer -- e.g. TCP
----
Internet Layer -- e.g. IP
----
Link Layer -- e.g. IEEE 802.2

The so-called OSI (Open Systems Interconnection) model defines seven layers. We’ll take a look at the application, transport, and Internet layers for the typical HTTP usage: HTTP, TCP, and IP. The layers below IP are the data link and physical layers. These are the layers that, e.g. implement Ethernet (Ethernet has a data link part and a physical part).

We will only look at the application, transport, and Internet layers, and in fact only look at one particular combination: HTTP running on top of TCP, which in turn runs on top of IP. This is the typical setup most of us use for our apps, day in and day out.

We hope that this will give you a more detailed understanding of how HTTP works under the hood, as well as what some common problems are, and how you can avoid them.

There are other ways than HTTP to send data through the Internet. One reason that HTTP has become so popular is that it will almost always work, even when the machine is behind a firewall.

Let’s start out at the lowest layer and take a look at IP, the Internet Protocol.

IP — Internet Protocol

The IP in TCP/IP is short for Internet Protocol. As the name suggests, it is one of the fundamental protocols of the Internet.

IP implements packet-switched networking. It has a concept of hosts which are machines. The IP protocol specifies how datagrams (packets) are sent between these hosts.

A packet is a chunk of binary data that has a source host and a destination host. An IP network will then simply transmit the packet from the source to the destination. One important aspect of IP is that packets are delivered using best effort. A packet may be lost along the way and never reach the destination. Or it may get duplicated and arrive multiple times at the destination.

Each host in an IP network has an address – the so-called IP address. Each packet contains the source and destination hosts’ addresses. The IP is responsible for routing datagrams: as the IP packet travels through the network, each node (host) that it travels through looks at the destination address in the packet to figure out in which direction the packet should be forwarded.

Today, most packages are still IPv4 (Internet Protocol version 4), where each IPv4 address is 32 bits long. They’re most often written in dotted-decimal notation, like so: 198.51.100.42

The newer IPv6 standard is slowly gaining traction. It has an address space: its addresses are 128 bits long. This allows for easier routing as the packets travel through the network. And since there are more available addresses, tricks such as network address translation are no longer necessary. IPv6 addresses are represented in the hexadecimal system and divided into eight groups separated by colons, e.g. 2001:0db8:85a3:0042:1000:8a2e:0370:7334.

The IP Header

An IP packet consists of a header and a payload.

The payload contains the actual data to be transmitted, while the header is metadata.

IPv4 Header

An IPv4 header looks like this:

IPv4 Header Format
Offsets  Octet    0                       1                       2                       3
Octet    Bit      0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31|
 0         0     |Version    |IHL        |DSCP            |ECN  |Total Length                                   |
 4        32     |Identification                                |Flags   |Fragment Offset                       |
 8        64     |Time To Live           |Protocol              |Header Checksum                                |
12        96     |Source IP Address                                                                             |
16       128     |Destination IP Address                                                                        |
20       160     |Options (if IHL > 5)                                                                          |

The header is 20 bytes long (without options, which are rarely used).

The most interesting parts of the header are the source and destination IP addresses. Aside from that, the version field will be set to 4 – it’s IPv4. And the protocol field specified which protocol the payload is using. TCP’s protocol number is 6. The total length field specified is the length of the entire packet – header plus payload.

Check Wikipedia’s article on IPv4 for all the details about the header and its fields.

IPv6 Header

IPv6 uses addresses that are 128 bits long. The IPv6 header looks like this:

Offsets  Octet    0                       1                       2                       3
Octet    Bit      0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31|
 0         0     |Version    |Traffic Class         |Flow Label                                                 |
 4        32     |Payload Length                                |Next Header            |Hop Limit              |
 8        64     |Source Address                                                                                |
12        96     |                                                                                              |
16       128     |                                                                                              |
20       160     |                                                                                              |
24       192     |Destination Address                                                                           |
28       224     |                                                                                              |
32       256     |                                                                                              |
36       288     |                                                                                              |

The IPv6 header has a fixed length of 40 bytes. It’s a lot simpler than IPv4 – a few lessons were learned in the years that have passed since IPv4.

The source and destination addresses are again the most interesting fields. In IPv6 the next header field specifies what data follows the header. IPv6 allows chaining of headers inside the packet. Each subsequent IPv6 header will also have a next header field until the actual payload is reached. When the next header field, for example, is 6 (TCP’s protocol number), the rest of the packet will be TCP data.

Again: Wikipedia’s article on IPv6 packets has a lot more detail.

Fragmentation

In IPv4, packets (datagrams) can get fragmented. The underlying transport layer will have an upper limit to the length of packet it can support. In IPv4, a router may fragment a packet if it gets routed onto an underlying data link for which the packet would otherwise be too big. These packets will then get reassembled at the destination host.

In IPv6, a router will drop the packet and send back a Packet Too Big message to the sender. The end points use this to figure out what the so-called maximum transfer unit (MTU) is. Only when the minimum payload size is too big for that MTU will IPv6 use fragmentation.

TCP — Transmission Control Protocol

One of the most common protocols to run on top of IP is, by far, TCP. It’s so common that the entire suite of protocols is often referred to as TCP/IP.

The IP protocol allows for sending single packets (datagrams) between two hosts. Packets are delivered best effort and may: reach the destination in a different order than the one in which they were sent, reach the destination multiple times, or never reach the destination at all.

TCP is built on top of IP. The Transmission Control Protocol provides reliable, ordered, error-checked delivery of a stream of data between programs. With TCP, an application running on one device can send data to an application on another device and be sure that the data arrives there in the same way that it was sent. This may seem trivial, but it’s really a stark contrast to how the raw IP layer works.

With TCP, applications establish connections between each other. A TCP connection is duplex and allows data to flow in both directions. The applications on either end do not have to worry about the data being split up into packets, or the fact that the packet transport is best effort. TCP guarantees that the data will arrive at the other end in pristine condition.

A typical use case of TCP is HTTP. Our web browser (application 1) connects to a web server (application 2). Once the connection is made, the browser can send a request through the connection, and the web server can send a response back through the same connection.

Multiple applications on the same host can use TCP simultaneously. To uniquely identify an application, TCP has a concept of ports. A connection between two applications has a source IP address and a source port on one end, and a destination IP address and a destination port at the other end. This pair of addresses, plus a port for either end, uniquely identifies the connection.

A web server using HTTPS will listen on port 443. The browser will use a so-called ephemeral port as the source port and then use TCP to establish a connection between the two address-port pairs.

TCP runs unmodified on top of both IPv4 and IPv6. The Protocol (IPv4) or Next Header (IPv6) field will be set to 6, which is the protocol number for TCP.

TCP Segments

The data stream that flows between hosts is cut up into chunks, which are turned into TCP segments. The TPC segment then becomes the payload of an IP packet.

Each TCP segment has a header and a payload. The payload is the actual data chunk to be transmitted. The TCP segment header first and foremost contains the source and destination port number – the source and destination addresses are already present in the IP header.

The header also contains sequence and acknowledgement numbers and quite a few other fields which are all used by TCP to manage the connection.

We’ll go into more detail about sequence number in a bit. It’s basically a mechanism to give each segment a unique number. The first segment has a random number, e.g. 1721092979, and subsequent segments increase this number by 1: 1721092980, 1721092981, and so on. The acknowledgement numbers allow the other end to communicate back to the sender regarding which segments it has received so far. Since TCP is duplex, this happens in both directions.

TCP Connections

Connection management is a central component of TCP. The protocol needs to pull a lot of tricks to hide the complexities of the unreliable IP layer. We’ll take a quick look at connection setup, the actual data flow, and connection termination.

The state transitions that a connection can go through are quite complex (c.f. TCP state diagram). But in most cases, things are relatively simple.

Connection Setup

In TCP, a connection is always established from one host to another. Hence, there are two different roles in connection setup: one end (e.g. the web server) is listening for connections, while the other end (e.g. our app) connects to the listening application (e.g. the web server). The server performs a so-called passive open – it starts listening. The client performs a so-called active open toward the server.

Connection setup happens through a three-way handshake. It works like this:

  1. The client sends a SYN to the server with a random sequence number, A
  2. The server replies with a SYN-ACK with an acknowledgment number of A+1 and a random sequence number, B
  3. The client sends an ACK to the server with an acknowledgement number of B+1 and a sequence number of A+1

SYN is short for synchronize sequence numbers. Once data flows between both ends, each TCP segment has a sequence number. This is how TCP makes sure that all parts arrive at the other end, and that they’re put together in the right order. Before communication can start, both ends need to synchronize the sequence number of the first segments.

ACK is short for acknowledgment. When a segment arrives at one of the ends, that end will acknowledge the receipt of that segment by sending an acknowledgment for the sequence number of the received segment.

If we run´:

curl -4 http://www.apple.com/contact/

this will cause curl to create a TCP connection to http://www.apple.com on port 80.

The server http://www.apple.com / 23.63.125.15 is listening on port 80. Our own address in the output is 10.0.1.6, and our ephemeral port is 52181 (this is a random, available port). The output from tcpdump(1) for the three-way handshake looks like this:

% sudo tcpdump -c 3 -i en3 -nS host 23.63.125.15
18:31:29.140787 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [S], seq 1721092979, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 743929763 ecr 0,sackOK,eol], length 0
18:31:29.150866 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [S.], seq 673593777, ack 1721092980, win 14480, options [mss 1460,sackOK,TS val 1433256622 ecr 743929763,nop,wscale 1], length 0
18:31:29.150908 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673593778, win 8235, options [nop,nop,TS val 743929773 ecr 1433256622], length 0

That’s a lot of information right there. Let’s step through this bit by bit.

On the very left-hand side we see the system time. This was run in the evening at 18:31. Next, IP tells us that these are IP packets.

Next we see 10.0.1.6.52181 > 23.63.125.15.80. This is the source and destination address-port pair. The first and third lines are from the client to the server, the second from the server to the client. tcpdump will simply append the port number to the IP address. 10.0.1.6.52181 means IP address 10.0.1.6, port 52181.

The Flags are flags in the TCP segment header: S for SYN, . for ACK, P for PUSH, and F for FIN. There are a few more we won’t see here. Note how these three lines have SYN, then SYN-ACK, then ACK: this is the three-way handshake.

The first line shows the client sending the random sequence number 1721092979 (A) to the server. The second line shows the server sending an acknowledgement for 1721092980 (A+1) and its random sequence number 673593777 (B). Finally, the third line shows the client acknowledging 673593778 (B+1).

Options

Another thing that happens during connection setup is for both ends to exchange additional options. In the first line, we see the client sending:

[mss 1460,nop,wscale 4,nop,nop,TS val 743929763 ecr 0,sackOK,eol]

and on the second line, the server is sending:

[mss 1460,sackOK,TS val 1433256622 ecr 743929763,nop,wscale 1]

The TS val / ecr are used by TCP to estimate the round-trip time (RTT). The TS val part is the time stamp of the sender, and the (ecr) is the timestamp echo reply, which is (generally) the last timestamp that the sender has received so far. TCP uses the round-trip time for its congestion-control algorithms.

Both ends are sending sackOK. This will enable Selective Acknowledgement. It switches the sequence numbers and acknowledgment number to use byte range instead of TCP segment numbers. And it then allows both ends to acknowledge receipt by byte range. This is outlined in section 3 of RFC 2018.

The mss option specified the Maximum Segment Size, which is the maximum number of bytes that this end is willing to receive in a single segment. wscale is the window scale factor that we’ll talk about in a bit.

Connection Data Flow

When the connection is created, both ends can send data to the other end. Each segment that is sent has a sequence number one larger than the previous segment. The receiving end will acknowledge packets as they are received by sending back segments with the corresponding ACK.

In theory this may looks like:

host A sends segment with seq 0
host A sends segment with seq 1
host A sends segment with seq 2     host B sends segment with ack 0
host A sends segment with seq 3     host B sends segment with ack 1
                                    host B sends segment with ack 2
                                    host B sends segment with ack 3

This mechanism happens in both directions. Host A will keep sending packets. As they arrive at host B, host B will send acknowledgements for these packets back to host A. But host A will keep sending packets without waiting for host B to acknowledge them.

TCP incorporates flow control and a lot of sophisticated mechanisms for congestion control. These are all about figuring out (A) if segments got lost and need to be retransmitted, and (B) if the rate at which segments are sent needs to be adjusted.

Flow control is about making sure that the sending side doesn’t send data faster than the receiver (at either end) can process it. The receiver sends a so-called receive window, which tells the sender how much more data the receiver can buffer. There are some subtle details we’ll skip, but in the above tcpdump output, we see a win 65535 and a wscale 4. The first is the window size, the latter a scale factor. As a result, the host at 10.0.1.6 says it has a receive window of 4·64 kB = 256 kB and the the host at 23.63.125.15 advertises win 14480 and wscale 1 i.e. roughly 14 kB. As either side receives data, it will send an updated receive window to the other end.

Congestion control is quite complex. The various mechanisms are all about figuring out at which rate data can be sent through the network. It’s a very delicate balance. On one hand, there’s the obvious desire to send data as fast as possible, but on the other hand, the performance will collapse dramatically when sending too much data. This is called congestive collapse and it’s a property inherent to packet-switched networks. When too many packets are sent, packets will collide with other packets and the packet loss rate will climb dramatically.

The congestion control mechanisms need to also make sure that they play well with other flows. Today’s congestion control mechanisms in TCP are outlined in detail in about 6,000 words in RFC 5681. The basic idea is that the sender side looks at the acknowledgments it gets back. This is a very tricky business and there are various tradeoffs to be made. Remember that the underlying IP packets can arrive out of order, not at all, or twice. The sender needs to estimate what the round-trip time is and use that to figure out if it should have received an acknowledgement already. Retransmitting packets is obviously costly, but not retransmitting causes the connection to stall for a while, and the load on the network is likely to be very dynamic. The TCP algorithms need to constantly adapt to the current situation.

The important thing to note is that a TCP connection is a very lively and flexible thing. Aside from the actual data flow, both ends constantly send hints and updates back and forth to continuously fine-tune the connection.

Because of this tuning, TCP connections that are short-lived can be very inefficient. When a connection is first created, the TCP algorithms still don’t know what the conditions of the network are. And toward the end of the connection lifetime, there’s less information flowing back to the sender, which therefore has a harder time estimating how things are moving along.

Above, we saw the first three segments between the client and the server. If we look at the remainder of the connection, it looks like this:

18:31:29.150955 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [P.], seq 1721092980:1721093065, ack 673593778, win 8235, options [nop,nop,TS val 743929773 ecr 1433256622], length 85
18:31:29.161213 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], ack 1721093065, win 7240, options [nop,nop,TS val 1433256633 ecr 743929773], length 0

The client at 10.0.1.6 sends the first segment with data length 85 (the HTTP request, 85 bytes). The ACK number is left at the same value, because no data has been received from the other end since the last segment.

The server at 23.63.125.15 then acknowledges the receipt of that data (but doesn’t send any data): length 0. Since the connection is using Selective acknowledgments, the sequence number and acknowledgment numbers are byte ranges: 1721092980 to 1721093065 is 85 bytes. When the other end sends ack 1721093065, that means: I have everything up to byte 1721093065.

This continues until all data has been sent:

18:31:29.189335 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673593778:673595226, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190280 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673595226:673596674, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190350 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673596674, win 8101, options [nop,nop,TS val 743929811 ecr 1433256660], length 0
18:31:29.190597 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673596674:673598122, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190601 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673598122:673599570, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190614 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673599570:673601018, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190616 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673601018:673602466, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190617 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673602466:673603914, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190619 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673603914:673605362, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190621 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673605362:673606810, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.190679 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673599570, win 8011, options [nop,nop,TS val 743929812 ecr 1433256660], length 0
18:31:29.190683 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673602466, win 7830, options [nop,nop,TS val 743929812 ecr 1433256660], length 0
18:31:29.190688 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673605362, win 7830, options [nop,nop,TS val 743929812 ecr 1433256660], length 0
18:31:29.190703 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673605362, win 8192, options [nop,nop,TS val 743929812 ecr 1433256660], length 0
18:31:29.190743 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673606810, win 8192, options [nop,nop,TS val 743929812 ecr 1433256660], length 0
18:31:29.190870 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [.], seq 673606810:673608258, ack 1721093065, win 7240, options [nop,nop,TS val 1433256660 ecr 743929773], length 1448
18:31:29.198582 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [P.], seq 673608258:673608401, ack 1721093065, win 7240, options [nop,nop,TS val 1433256670 ecr 743929811], length 143
18:31:29.198672 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673608401, win 8183, options [nop,nop,TS val 743929819 ecr 1433256660], length 0
Connection Termination

Finally the connection is torn down (or terminated). Each end will send a FIN flag to the other end to tell it that it’s done sending. This FIN is then acknowledged. When both ends have sent their FIN flags and they have been acknowledged, the connection is fully closed:

18:31:29.199029 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [F.], seq 1721093065, ack 673608401, win 8192, options [nop,nop,TS val 743929819 ecr 1433256660], length 0
18:31:29.208416 IP 23.63.125.15.80 > 10.0.1.6.52181: Flags [F.], seq 673608401, ack 1721093066, win 7240, options [nop,nop,TS val 1433256680 ecr 743929819], length 0
18:31:29.208493 IP 10.0.1.6.52181 > 23.63.125.15.80: Flags [.], ack 673608402, win 8192, options [nop,nop,TS val 743929828 ecr 1433256680], length 0

Note how on the second line, 23.63.125.15 sends its FIN, and at the same time acknowledges the other end’s FIN with an ACK (dot), all in a single segment.

HTTP — Hypertext Transfer Protocol

The World Wide Web of interlinked hypertext documents and a browser to browse this web started as an idea set forward in 1989 at CERN. The protocol to be used for data communication was the Hypertext Transfer Protocol, or HTTP. Today’s version is HTTP/1.1, which is defined in RFC 2616.

Request and Response

HTTP uses a simple request and response mechanism. When we enter http://www.apple.com/ into Safari, it send an HTTP request to the server at http://www.apple.com. The server in turn sends back a (single) response which contains the document that was requested.

There’s always a single request and a single response. And both requests and responses follow the same format. The first line is the request line (request) or status line (response). This line is followed by headers. The headers end with an empty line. After that, an empty line follows the optional message body.

A Simple Request

When Safari loads the HTML for http://www.objc.io/about.html, it sends an HTTP request to http://www.objc.io with the following content:

GET /about.html HTTP/1.1
Host: www.objc.io
Accept-Encoding: gzip, deflate
Connection: keep-alive
If-None-Match: "a54907f38b306fe3ae4f32c003ddd507"
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
If-Modified-Since: Mon, 10 Feb 2014 18:08:48 GMT
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.74.9 (KHTML, like Gecko) Version/7.0.2 Safari/537.74.9
Referer: http://www.objc.io/
DNT: 1
Accept-Language: en-us

The first line is the request line. It specifies the desired action: GET. This is called the HTTP method. Next up the path of the resources that this action is supposed to be made on: /about.html, and finally the HTTP version. In this case, we want to get the document.

Next, we have 10 lines with 10 HTTP headers. These are followed by an empty line. There’s no message body in this request.

The headers have very varying purposes. They convey additional information to the web server. Wikipedia has a nice list of common HTTP header fields. The first Host: http://www.objc.io header tells the server what server name the request is meant for. This mandatory request header allows the same physical server to serve multiple domain names.

Let’s look at a few common ones:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us

This tells the server what media type Safari would like to receive. A server may be able to send a response in various formats. The text/html strings are Internet media types, sometimes also known as MIME types or Content-types. The q=0.9 allows Safari to convey a quality factor which it associates with the given media types. Accept-Language tells the server which languages Safari would prefer. Again, this lets the server pick the matching languages, if available.

Accept-Encoding: gzip, deflate

With this header, Safari tells the server that the response body can be sent compressed. If this header is not set, the server must send the data uncompressed. Particularly for text (such as HTML), compression rates can dramatically reduce the amount of data that has to be sent.

If-Modified-Since: Mon, 10 Feb 2014 18:08:48 GMT
If-None-Match: "a54907f38b306fe3ae4f32c003ddd507"

These two are due to the fact that Safari already has the resulting document in its cache. Safari tells the server only to send it if it has either changed since February 10, or if its so-called ETag doesn’t match a54907f38b306fe3ae4f32c003ddd507.

The User-Agent header tells the server what kind of client is making the request.

A Simple Response

In response to the above, the server responds with:

HTTP/1.1 304 Not Modified
Connection: keep-alive
Date: Mon, 03 Mar 2014 21:09:45 GMT
Cache-Control: max-age=3600
ETag: "a54907f38b306fe3ae4f32c003ddd507"
Last-Modified: Mon, 10 Feb 2014 18:08:48 GMT
Age: 6
X-Cache: Hit from cloudfront
Via: 1.1 eb67cb25620df959ba21a943fbc49ef6.cloudfront.net (CloudFront)
X-Amz-Cf-Id: dDSBgR86EKBemW6el-pBI9kAnuYJEaPQYEqGmBnilD12CbixCuZYVQ==

The first line is the so-called status line. It contains the HTTP version, followed by a status code (304) and a status message.

HTTP defines a list of status codes and their meanings. In this case, we’re receiving 304, which means that the resource we requested hasn’t been modified.

The response doesn’t contain any body message. It simply tells the receiver: Your version is up to date.

Caching Turned Off

Let’s do another request with curl:

% curl http://www.apple.com/hotnews/ > /dev/null

curl doesn’t use a local cache. The entire request will now look like this:

GET /hotnews/ HTTP/1.1
User-Agent: curl/7.30.0
Host: www.apple.com
Accept: */*

This is quite similar to what Safari was requesting. This time, there’s no If-None-Match header and the server will have to send the document.

Note how curl announces that it will accept any kind of media format: (Accept: */*).

The response from http://www.apple.com looks like this:

HTTP/1.1 200 OK
Server: Apache
Content-Type: text/html; charset=UTF-8
Cache-Control: max-age=424
Expires: Mon, 03 Mar 2014 21:57:55 GMT
Date: Mon, 03 Mar 2014 21:50:51 GMT
Content-Length: 12342
Connection: keep-alive

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
<head>
    <meta charset="utf-8" />

and continues on for quite a while. It now has a response body which contains the HTML document.

The response from Apple’s server contains a status code of 200, which is the standard response for successful HTTP requests.

Apple’s server also tells us that the response’s media type is text/html; charset=UTF-8. The Content-Length: 12342 tells us what the length of the message body is.

HTTPS — HTTP Secure

Transport Layer Security is a cryptographic protocol that runs on top of TCP. It allows for two things: both ends can verify the identity of the other end, and the data sent between both ends is encrypted. Using HTTP on top of TLS gives you HTTP Secure, or simply, HTTPS.

Simply using HTTPS instead of HTTP will give you a huge improvement in security. There are some additional steps that you may want to take, though, both of which will additionally improve the security of your communication.

TSL 1.2

You should set the TLSMinimumSupportedProtocol to kTLSProtocol12 to require TLS version 1.2 if your server supports that. This will make man-in-the-middle attacks more difficult.

Certificate Pinning

There’s little point in the data we send being encrypted if we can’t be certain that the other end we’re talking to is actually who we think it is. The server’s certificate tells us who the server is. Only allowing a connection to a very specific certificate is called certificate pinning.

When a client makes a connection over TLS to a server, the operating system will decide if it thinks the server’s certificate is valid. There are a few ways this can be circumvented, most notably by installing certificates onto the iOS device and marking them as trusted. Once that’s been done, it’s trivial to perform a man-in-the-middle attack against your app.

To prevent this from happening (or at least make it extremely difficult), we can use a method called certificate pinning. When then TLS connection is set up, we inspect the server’s certificate and check not only that it’s to be considered valid, but also that it is the certificate that we expect our server to have. This only works if we’re connecting to our own server and hence can coordinate an update to the server’s certificate with an update to the app.

To do this, we need to inspect the so-called server trust during connection. When an NSURLSession creates a connection, the delegate receives a -URLSession:didReceiveChallenge:completionHandler: call. The passed NSURLAuthenticationChallenge has a protectionSpace property, which is an instance of NSURLProtectionSpace. This, in turn, has a serverTrust property.

The serverTrust is a SecTrustRef object. The Security framework has various methods to query the SecTrustRef object. The AFSecurityPolicy from the AFNetworking project is a good starting point. As always, when you build your own security-related code, have someone review it carefully. You don’t want to have a goto fail; bug in this part of your code.

Putting the Pieces Together

Now that we know a bit about how all the pieces (IP, TCP, and HTTP) work, there are a few things we can do and be aware of.

Efficiently Using Connections

There are two aspects of a TCP connection that are problematic: the initial setup, and the last few segments that are pushed across the connection.

Setup

The connection setup can be very time consuming. As mentioned above, TCP needs to do a three-way handshake. There’s not a lot of data that needs to be sent back and forth. But, particularly when on a mobile network, the time a packet takes to travel from one host (a user’s iPhone) to another host (a web server) can easily be around 250 ms – a quarter of a second. For a three-way handshake, we’ll often spend 750 ms just to establish the connection, before any payload data has been sent.

In the case of HTTPS, things are even more dramatic. With HTTPS we have HTTP running on top of TLS, which in turn runs on top of TCP. The TCP connection will still do its three-way handshake. Next up, the TLS layer does another three-way handshake. Roughly speaking, an HTTPS connection will thus take twice as long as a normal HTTP connection before sending any data . If the round-trip time is 500 ms (250 ms end-to-end), that adds up to 1.5 seconds.

This setup time is costly, regardless of if the connection will transfer a lot or just a small amount of data.

Another aspect of TCP affects connections on which we expect to transfer a larger amount of data. When sending segments into a network with unknown conditions, TCP needs to probe the network to determine the available capacity. In other words: it takes TCP a while to figure out how fast it can send data over the network. Only once it has figured this out can it send the data at the optimal speed. The algorithm that is used for this is called slow-start. On a side note, it’s worth pointing out that the slow-start algorithm doesn’t perform well on networks with poor data link layer transmission quality, as is often the case for wireless networks.

Tear Down

Another problem arises toward the end of the data transfer. When we do an HTTP request for a resource, the server will keep sending TCP segments to our host, and the host will respond with ACK messages as it receives the data. If a single packet is lost along the way, the server will not receive an ACK for that packet. It can therefore notice that the packet was lost and do what’s called a fast retransmit.

When a packet is lost, the following packet will cause the receiver to send an ACK identical to the last ACK it sent. The receiver will hence receive a duplicate ACK. There are several network conditions that can cause a duplicate ACK even without packet loss. The sender therefore only performs a fast retransmit when it receives three duplicate ACKs.

The problem with this is at the end of the data transmission. When the sender stops sending segments, the receiver stops sending ACKs back. And there’s no way for the fast-retransmit algorithm to detect if a packet is lost within the last four segments being sent. On a typical network, that’s equivalent to 5.7 kB of data. Within the last 5.7 kB, the fast retransmit can’t do its job. If a packet is lost, TCP has to fall back to a more patient algorithm to detect packet loss. It’s not uncommon for a retransmit to take several seconds in such a case.

Keep-Alive and Pipelining

HTTP has two strategies to counter these problems. The simplest is called HTTP persistent connection, sometimes also known as keep-alive. HTTP will simply reuse the same TCP connection once a single request-response pair is done. In case of HTTPS, the same TLS connection will be re-used:

open connection
client sends HTTP request 1 ->
                            <- server sends HTTP response 1
client sends HTTP request 2 ->
                            <- server sends HTTP response 2
client sends HTTP request 3 ->
                            <- server sends HTTP response 3
close connection

The second step is to use HTTP pipelining, which allows the client to send multiple requests over the same connection without waiting for the response from previous requests. The requests can be sent in parallel with responses being received, but the responses are still sent back to the client in the order that the requests were sent – it’s still a first-in-first-out principle.

Slightly simplified, this would look like:

open connection
client sends HTTP request 1 ->
client sends HTTP request 2 ->
client sends HTTP request 3 ->
client sends HTTP request 4 ->
                            <- server sends HTTP response 1
                            <- server sends HTTP response 2
                            <- server sends HTTP response 3
                            <- server sends HTTP response 4
close connection

Note, however, that the server may start sending the response at any time, and doesn’t have to wait until all requests have been received.

With this, we’re able to use TCP in a more efficient way. We’re now only doing the handshake at the very beginning, and since we keep using the same connection, TCP can do a much better job of using the available bandwidth. The TCP congestion control algorithms will also do a much better job. Finally, the problem with the fast retransmit failing to work on the last four segments will only affect the last four segments of the entire connection, and not the last four segments of each request-response, as before.

The improvements of using HTTP pipelining can be quite dramatic over high-latency connections – which is what you have when your iPhone is not on Wi-Fi. In fact, there’s been some research that suggests that there’s no additional performance benefit to using SPDY over HTTP pipelining on mobile networks.

When communication with a server, RFC 2616 recommends using two connections when HTTP pipelining is enabled. According to these guidelines, this will result in the best response times and avoid congestion. There’s no performance benefit to additional connections.

Sadly, there are still some servers that don’t support pipelining. You should try to enable it, however, and check if your particular server does support it. Chances are, it does. But because some servers don’t, HTTP pipelining is turned off by default for NSURLSession. Make sure to set HTTPShouldUsePipelining to YES on the NSURLSessionConfiguration if you’re able to use pipelining.

Timeout

We have all used apps while being on a slow network. Many of these apps give up after a timeout, say 15 seconds. That is really bad app design. It’s probably good to give the user feedback and say, “Hey, your network seems to be slow. This may take some time.” But as long as there’s a connection, even if it’s bad, TCP ensure that the request will make it to the other end and that the response will make it back. It may just take a while.

Or look at it another way: If we’re on a slow network, the request-response round-trip time may take 17 seconds to complete. If we’re stopping after 15 seconds, the user will be unable to perform the desired action if he or she is patient. If the user is on a slow network, he or she will know that things will take a while (we’ll send a notification), and if the user is willing to be patient, we shouldn’t prevent him or her from using the app.

There’s a misconception that restarting the (HTTP) request will fix the problem. That is not the case. Again, TCP will resend those packets that need resending on its own.

The right approach is this: When we send off a URL request, we set a timer for 10 seconds. Once we get the response back, we invalidate the timer. If the timer fires before we get the response back, we’ll show some UI: “Please be patient, the network appears to be slow right now.” Depending on the app, we might want to give the user an option to cancel the action. But we shouldn’t make that decision for the user.

Caching

Note how in our first example, we send a:

If-None-Match: "a54907f38b306fe3ae4f32c003ddd507"

line to the server, letting it know that we already have a resource locally and only want the server to send the resource to us if it has a newer version. If you’re building your own communication with your server, try to make use of this mechanism. It can dramatically speed up communication if used correctly. The mechanism is called HTTP ETag.

Going to extremes, remember: “The fastest request is the one never made.” When sending a request to a server, even on a very good network, requesting a small amount of data, and a very fast server, you’re unlikely to have a response back in less than 50 ms. And that’s just one request. If there’s a way for you to create that data locally in 50 ms or less, don’t make that request.

Cache resources locally whenever you think that they’ll stay valid for some time. Check the Expires header or rely on the caching mechanisms of NSURLSession by using the default NSURLRequestUseProtocolCachePolicy.

Summary

Sending a request over HTTP with NSURLSession is very straightforward. But there are a lot of technologies involved in making that request. Knowing how those pieces work will allow you to optimize the way you’re using HTTP requests. Our users expect apps that perform well under any condition. Understanding how IP, TCP, and HTTP work makes it easier to deliver on those expectations.