I\'ve been reading Aleph One\'s paper on Smashing the Stack for Fun and Profit.

Question

I've been reading Aleph One's paper on Smashing the Stack for Fun and Profit. I wrote down example1.c from his paper, modified it a bit to see what the stack looks like on my system.

I'm running Ubuntu (64-bit) on a VM on an Intel i5 M 480.

The paper says that a stack will have the following structure. It also says that the word size is 4 bytes. I read up on word sizes and determined that on a 64-bit OS that is not "long-enabled" the word size is still 32 bits or 4 bytes.

enter image description here

However, when I run my custom code:

void function(int a, int b, int c) {
    char buffer1[5];
    char buffer2[10];
    memset(buffer1, "ª", sizeof(buffer1));
}

void main() {
    function(1, 2, 3);
}
I do not get the same stack structure as the paper. Yes, I'm aware that the paper was published in 1998 but I haven't found any article on the internet stating that the stack structure has been modified greatly. Here's what my stack looks like (I'm also uploading GDB screenshots for verification, in case I've misinterpreted the stack):

    Lower memory                                                Higher memory
    -------------------------------------------------------------------------
    | int c   | int b   | int a   | buffer1 | buffer2 | RBP     | RET     |
    | 4 bytes | 4 bytes | 4 bytes | 16 bytes | 16 bytes | 8 bytes | 8 bytes |
    -------------------------------------------------------------------------
enter image description here enter image description here

Now for my questions:

Why has the stack structure changed?
What is with the extra space given to buffer1 and buffer2? According to the paper they should have only 8 bytes and 12 bytes allotted. However, buffer2 gets an extra 6 bytes and only then does buffer1 begin and even buffer1 is allotted 16 bytes. Am I missing something here? I read about slack space being given as a protective mechanism, is this it?

Explanation / Answer

The difference is mostly explained by the difference of ABI :

the paper explains what happens in x86 (32 bit);

you are using x86_64.

In x86n the x86 ABI is used

The parameters are passed on the stack by the caller. This means that the 3 values on the top will be a, b and c.

The call instruction pushes the IP (the ret field).

Usually, the caller frame pointer is pushed by the callee (the sfp field).

All those fields are fixed on this architecture by the ABI.

The lower part of the stack (local variables, saved registers, etc.) is not fixed by the ABI and the compiler can decide how it will use it.

Lower memory Higher memory
-------------------------------------------------------------------------   
| buffer2 | buffer2 | sfp | ret | a | b | c |
-------------------------------------------------------------------------
<----------(3)------><--(2)--><--------------------(1)------------------>

(1): Pushed by the caller for the callee, fixed by the ABI.
(2): Pushed by the callee, fixed by the ABI.
Technically, this is optional as for x86 but it is generally used.
(3): Pushed by the callee, in any order.
In x86_64, the AMD-64/x86_64 ABI is used:

Parameters are usually passed by registers: the a, b, c variables that you find on the stack are copies of the parameters made by the callee (because you probably did not enable optimisations). This is why they are lower in the stack than the return address. This means that the compiler if free to place them in any order (and does not need to have them on the stack at all).

Moreover, the code is usually not compiled with frame pointers: the push rbp; mov rbp, rsp is usually omitted.

Lower memory Higher memory
-------------------------------------------------------------------------   
| int c | int b | int a | buffer1 | buffer2 | RBP | RET |
| 4 bytes | 4 bytes | 4 bytes | 16 bytes | 16 bytes | 8 bytes | 8 bytes |
-------------------------------------------------------------------------
<-------------(2)--------------------------------------------> <---(1)-->

(1): Pushed by the caller for the callee, fixed by the ABI.
(2): Pushed by the callee, in any order.
In summary:

On x86, the parameters are pushed on the stack by the caller: their locations on the stack is fixed by the ABI. They are higher than the return address because they are pushed by the caller.

On x86_64, the parameters are passed in registers (unless you have too many of them). The callee is free to push them on the stack if it needs to in any order. As they are pushed by the callee they are lower than the return address on the stack (and can be intermixed with local variables, saved registers).

In both ABIs, using %rbp as frame base is optional but it is usually used for x86 and often not used in x86_64.

Related Questions

Navigate

• Browse (All)
• Browse I
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.