Confused about context switch

[KrotovOSdev]

Hello, forum!
I were trying to start with my first device driver but firstly, of course, I have to implement scheduler. I have scheduling algorithm which works as it should (I'm not completely sure but...). The only problem for me is Context switching.
I was following the [wiki]Brendan's Multi-tasking Tutorial[/wiki] and I cant make context switch work properly. Now my code looks like this

Code: Select all

section .text
global switch_context

switch_context:
    push ebx
    push esi
    push edi
    push ebp
 
    mov edi,[now_executed]    ;edi = address of the previous task's "thread control block"
    mov [edi + TCB.esp],esp         ;Save ESP for previous task's kernel stack in the thread's TCB
 
    ;Load next task's state
 
    mov esi,[esp+(4+1)*4]         ;esi = address of the next task's "thread control block" (parameter passed on stack)
    mov [now_executed],esi    ;Current task's TCB is the next task TCB
 
    mov esp,[esi + TCB.esp]         ;Load ESP for next task's kernel stack from the thread's TCB
    mov eax,[esi + TCB.pgd]         ;eax = address of page directory for next task
    mov ebx,[esi + TCB.esp0]        ;ebx = address for the top of the next task's kernel stack
    mov [TSS + TSS_struc.esp0],ebx   ;Adjust the ESP0 field in the TSS (used by CPU for for CPL=3 -> CPL=0 privilege level changes)
    mov ecx,cr3                   ;ecx = previous task's virtual address space
 
    cmp eax,ecx                   ;Does the virtual address space need to being changed?
    je .doneVAS                   ; no, virtual address space is the same, so don't reload it and cause TLB flushes
    mov cr3,eax                   ; yes, load the next task's virtual address space
.doneVAS:
    pop ebp
    pop edi
    pop esi
    pop ebx
 
    ret                           ;Load next task's EIP from its kernel stack

TSS and now_executed are pointers to TSS and task structures. This is how I call this function

Code: Select all

switch_context(&(task));

, where task is TaskStructure. But this context switch function throws exception "Invalid opcode". Where is the problem? My idea is that I prepare stack for task wrong. Here is the code:

Code: Select all

void prepare_task(task_struct_t task, uint32_t eip) {
    uint32_t esp;
    asm volatile (
        "movl %%esp, %0"
        : "=r"(esp)
    );

    asm volatile (
        "mov %0, %%esp\n"
        "pushl %1\n"
        "push %%ebp\n"
        "push %%edi\n"
        "push %%esi\n"
        "push %%ebx\n"
        "movl %2, %%esp\n"
        :
        : "r"(task.esp), "r"(eip), "r"(esp)
    );
}

So I'm totally confused about this. Thank you for reply.

[Octocontrabass]

The problem is indeed your prepare_task() function. You don't need inline assembly. You do need to modify the task's stack pointer. If you want this function to update the stack pointer, you need to pass a pointer to the task structure.

Something like this?

Code: Select all

void prepare_task(task_struct_t * task, uint32_t eip) {
    uint32_t * stack = (uint32_t *)task->esp;
    stack[-1] = eip;
    task->esp -= 5 * sizeof( uint32_t );
}

Also, this isn't a problem, but you could simplify switch_context() a bit. It only needs to push the four callee-saved registers, save ESP, load ESP, pop the four registers, and return. The other things like updating CR3 and TSS.ESP0 can be done before you call it.

[KrotovOSdev]

The problem is indeed your prepare_task() function. You don't need inline assembly. You do need to modify the task's stack pointer. If you want this function to update the stack pointer, you need to pass a pointer to the task structure.

Something like this?

Code: Select all

void prepare_task(task_struct_t * task, uint32_t eip) {
    uint32_t * stack = (uint32_t *)task->esp;
    stack[-1] = eip;
    task->esp -= 5 * sizeof( uint32_t );
}

Also, this isn't a problem, but you could simplify switch_context() a bit. It only needs to push the four callee-saved registers, save ESP, load ESP, pop the four registers, and return. The other things like updating CR3 and TSS.ESP0 can be done before you call it.

Still have the same problem. Maybe I miss some important thing? If this can help, here is my code for preparing kernel task:

Code: Select all

task_struct_t create_kernel_task() {
    uint32_t esp, eip;
    asm volatile (
        "movl %%esp, %0\n"
        "call get_eip\n"
        "get_eip:\n"
        "pop %1\n"
        : "=r"(esp), "=r"(eip)
    );

    task_struct_t task = create_task(esp, TASK_EXEC_KERNEL);
    task.memory_map.pgd = page_directory;
    task.state = TASK_STATE_WAITING;
    task.exec_mode = TASK_EXEC_KERNEL;

    prepare_task(task, eip);
    return task;
}

[kzinti]

It looks like you are setting the eip of the new task to the current task's one. Are you trying to implement fork()? You will need to also copy the stack and a bunch of other things, it might be better to start with a simpler approach (i.e. don't try to fork).

For example, I start execution at a static function named "Task::Entry()". Here is what my setup looks like:

Code: Select all

void Task::Initialize(EntryPoint entryPoint, const void* args)
{
    const char* stack = (char*)GetStack();

    // We use an InterruptContext to "return" to the task's entry point. The reason we can't only use a CpuContext
    // is that we need to be able to set arguments for the entry point. These need to go in registers (rdi, rsi, rdx)
    // that aren't part of the CpuContext.
    constexpr auto interruptContextSize = sizeof(InterruptContext);
    stack = stack - mtl::AlignUp(interruptContextSize, 16);

    const auto interruptContext = (InterruptContext*)stack;
    interruptContext->rip = (uintptr_t)Task::Entry;                    // "Return" to Task::Entry
    interruptContext->cs = (uint64_t)Selector::KernelCode;             // "Return" to kernel code
    interruptContext->rflags = mtl::EFLAGS_RESERVED;                   // Start with interrupts disabled
    interruptContext->rsp = (uintptr_t)(stack + interruptContextSize); // Required by iretq
    interruptContext->ss = (uint64_t)Selector::KernelData;             // Required by iretq
    interruptContext->rdi = (uintptr_t)this;                           // Param 1 for Task::Entry
    interruptContext->rsi = (uintptr_t)entryPoint;                     // Param 2 for Task::Entry
    interruptContext->rdx = (uintptr_t)args;                           // Param 3 for Task::Entry

    // Setup a task switch interruptContext to simulate returning from an interrupt.
    stack = stack - sizeof(CpuContext);

    const auto cpuContext = (CpuContext*)stack;
    cpuContext->rip = (uintptr_t)InterruptExit;

    m_context = cpuContext;
}

void Task::Entry(Task* task, EntryPoint entryPoint, const void* args)
{
    task->m_state = TaskState::Running;
    entryPoint(task, args);

    // TODO: die
    for (;;)
        ;
}

[Octocontrabass]

If this can help, here is my code for preparing kernel task:

Why do you want the new task to start in the middle of your create_kernel_task() function? Why do you want the new task to use the same stack as the current task?

[KrotovOSdev]

If this can help, here is my code for preparing kernel task:

Why do you want the new task to start in the middle of your create_kernel_task() function? Why do you want the new task to use the same stack as the current task?

Create kernel task is a function which creates task function but not starts it.

[thewrongchristian]

Read, and understand, PORTABLE MULTITHREADING, which is the basis of GNU PTH.

It is a user level threading library, gives details on how it creates its initial thread context, in a mostly portable manner.

Basically if using the setjmp/longjmp method, in the creating thread, you temporarily switch to the stack of the thread being created, save some state (including the stack pointer on the new stack) using setjmp (which will return 0), then switch back to the old stack.

Then, when you want to actually switch to the new thread, you save the current thread state using setjmp, then longjump using the jmp_buf setup above in the new thread, and your code will now be running on the new stack, in the bootstrap function, returning != 0 from setjmp. That is then your signal to jump to the new thread code.

Once the initial thread context is created, switching threads is then quite simple, using existing C setjmp primitives (or POSIX context primitives, in the paper). A task switch becomes:

Code: Select all

  if (setjmp(currentthread->context)==0) {
    longjmp(nextthread->context, 1);
  }

I used this idea as the basis of my kernel threads. All kernel thread switching is implemented using setjmp/longjmp, which saves/restores the compiler visible state. That is all you need in the kernel thread, any user visible state such as address space (cr3) or the kernel stack in the TSS can be managed separately, and your interrupt handlers should already save the user level register state on the kernel stack. Kernel esp and cr3 management can be in separate code, as suggested by @Octocontrabass.

All you need then is the architecture specific code to execute your thread bootstrap code with some arbitrary stack pointer.

[KrotovOSdev]

Read, and understand, PORTABLE MULTITHREADING, which is the basis of GNU PTH.

It is a user level threading library, gives details on how it creates its initial thread context, in a mostly portable manner.

Basically if using the setjmp/longjmp method, in the creating thread, you temporarily switch to the stack of the thread being created, save some state (including the stack pointer on the new stack) using setjmp (which will return 0), then switch back to the old stack.

Then, when you want to actually switch to the new thread, you save the current thread state using setjmp, then longjump using the jmp_buf setup above in the new thread, and your code will now be running on the new stack, in the bootstrap function, returning != 0 from setjmp. That is then your signal to jump to the new thread code.

Once the initial thread context is created, switching threads is then quite simple, using existing C setjmp primitives (or POSIX context primitives, in the paper). A task switch becomes:

Code: Select all

  if (setjmp(currentthread->context)==0) {
    longjmp(nextthread->context, 1);
  }

I used this idea as the basis of my kernel threads. All kernel thread switching is implemented using setjmp/longjmp, which saves/restores the compiler visible state. That is all you need in the kernel thread, any user visible state such as address space (cr3) or the kernel stack in the TSS can be managed separately, and your interrupt handlers should already save the user level register state on the kernel stack. Kernel esp and cr3 management can be in separate code, as suggested by @Octocontrabass.

All you need then is the architecture specific code to execute your thread bootstrap code with some arbitrary stack pointer.

It looks like that now everything works fine even without longjump/setjump. Despithe this I'll try to understand this mechanisms.

[Octocontrabass]

Create kernel task is a function which creates task function but not starts it.

Yes. It sets the new task's EIP to point to the middle of create_kernel_task(), so that is the code the new task will begin executing. Why are you setting the new task's EIP that way? What code do you want the new task to execute?

[KrotovOSdev]

Create kernel task is a function which creates task function but not starts it.

Yes. It sets the new task's EIP to point to the middle of create_kernel_task(), so that is the code the new task will begin executing. Why are you setting the new task's EIP that way? What code do you want the new task to execute?

That was the first problem. I've moved label to the end of kernel initialization process but now I have another problem - kernel stack overflow. I think it might be connected with IRQ0 handler and I have to make handler function static.
In my case, kernel and scheduler use one stack. Is this a problem or not?

[kzinti]

Schedulers don't use stacks. Tasks use stacks. Each task needs its own stack. When you create a new task, you need to allocate a new stack for it and initialize %esp to point to it.

[Octocontrabass]

I've moved label to the end of kernel initialization process

That still doesn't sound right...

In my case, kernel and scheduler use one stack. Is this a problem or not?

Each thread needs its own stack. Interrupt handlers can use the stack from whichever thread they interrupt. When your scheduler is called from an interrupt handler, it can use the same stack as the interrupt handler.

You will probably want to call your scheduler without an interrupt handler sometimes, so don't design your scheduler to require interrupts.

[KrotovOSdev]

I've moved label to the end of kernel initialization process

That still doesn't sound right...

In my case, kernel and scheduler use one stack. Is this a problem or not?

Each thread needs its own stack. Interrupt handlers can use the stack from whichever thread they interrupt. When your scheduler is called from an interrupt handler, it can use the same stack as the interrupt handler.

You will probably want to call your scheduler without an interrupt handler sometimes, so don't design your scheduler to require interrupts.

I mean scheduler is a part of my kernel and it uses kernel stack.

[Octocontrabass]

As long as each thread has its own kernel stack, that's fine.

[KrotovOSdev]

After I moved EIP to the end of initialization process, it just overflows my stack. I think the problem may be connected with IRQ0 handler which never returns. Can it be so?