What is the reason apic interrupt is not working in application processor?

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
User avatar
gamingjam60
Member
Member
Posts: 42
Joined: Sat Aug 24, 2024 10:06 pm
Libera.chat IRC: gamingjam60
Location: India
GitHub: https://github.com/baponkar
Contact:
Contact gamingjam60

What is the reason apic interrupt is not working in application processor?

Post by gamingjam60 »

I am using Limine bootloader to load kernel in my x86 architecture based 64 bit system.

To start multiprocessor along with apic interrupt I have done following.

for bootstrap cpu
1. Disable PIC
2. Initialize ACPI
3. Initialiize GDT , TSS
4. Enable IOAPIC
5. Initialize TSC
6. Initialize APIC Interrupts
7. IOAPIC Route IRQ
8. Initialize Keybooard Interrupt
9. Initialize APIC Timer

Similarly for application cpu
1. Disable PIC
2. Initialize ACPI
3. Initialiize GDT , TSS
4. Enable IOAPIC
5. Initialize TSC
6. Initialize APIC Interrupts
7. Initialize Keybooard Interrupt
8. Initialize APIC Timer

Here my full code:

Code: Select all

cpu_data_t cpu_datas[MAX_CPUS];  // Array indexed by CPU ID (APIC ID)

extern struct limine_smp_response *smp_response;

extern madt_t *madt;


void start_bootstrap_cpu_core() {

    printf("\n[Info] Starting bootstrap CPU core...\n");

    if (smp_response == NULL) {
        printf("[Error] SMP response is null!\n");
        return;
    }
    
    if (smp_response->cpu_count == 0) {
        printf("[Error] No CPUs found!\n");
        return;
    }

    uint32_t bsp_lapic_id = smp_response->bsp_lapic_id;
    
    disable_pic();
    init_acpi();
    enable_ioapic_mode();
    parse_madt(madt);

    init_paging();

    init_tsc();

    gdt_tss_init();

    // Enable the Interrupts
    init_bootstrap_interrupt(bsp_lapic_id);
    init_apic_interrupt();
    int_syscall_init();

    init_apic_timer(100);

    initKeyboard();
   
    // Route IRQs to the bootstrap core no need to route them to other cores
    uint32_t bsp_flags = (0 << 8) | (0 << 13) | (0 << 15);
    
    ioapic_route_irq(0, bsp_lapic_id, 32, bsp_flags);      // Route IRQ 1 to current LAPIC ID with vector 33
    ioapic_route_irq(1, bsp_lapic_id, 33, bsp_flags);      // Route IRQ 1 to current LAPIC ID with vector 33
    ioapic_route_irq(2, bsp_lapic_id, 34, bsp_flags);      // Route IRQ 2 to current LAPIC ID with vector 34
    ioapic_route_irq(3, bsp_lapic_id, 35, bsp_flags);      // Route IRQ 3 to current LAPIC ID with vector 35
    ioapic_route_irq(4, bsp_lapic_id, 36, bsp_flags);      // Route IRQ 4 to current LAPIC ID with vector 36
    ioapic_route_irq(5, bsp_lapic_id, 37, bsp_flags);      // Route IRQ 5 to current LAPIC ID with vector 37
    ioapic_route_irq(6, bsp_lapic_id, 38, bsp_flags);      // Route IRQ 6 to current LAPIC ID with vector 38
    ioapic_route_irq(7, bsp_lapic_id, 39, bsp_flags);      // Route IRQ 7 to current LAPIC ID with vector 39
    ioapic_route_irq(8, bsp_lapic_id, 40, bsp_flags);      // Route IRQ 8 to current LAPIC ID with vector 40
    ioapic_route_irq(9, bsp_lapic_id, 41, bsp_flags);      // Route IRQ 9 to current LAPIC ID with vector 41
    ioapic_route_irq(10, bsp_lapic_id, 42, bsp_flags);     // Route IRQ 10 to current LAPIC ID with vector 42
    ioapic_route_irq(11, bsp_lapic_id, 43, bsp_flags);     // Route IRQ 11 to current LAPIC ID with vector 43
    ioapic_route_irq(12, bsp_lapic_id, 44, bsp_flags);     // Route IRQ 12 to current LAPIC ID with vector 44
    ioapic_route_irq(13, bsp_lapic_id, 45, bsp_flags);     // Route IRQ 13 to current LAPIC ID with vector 45
    ioapic_route_irq(14, bsp_lapic_id, 46, bsp_flags);     // Route IRQ 14 to current LAPIC ID with vector 46
    ioapic_route_irq(15, bsp_lapic_id, 47, bsp_flags);     // Route IRQ 15 to current LAPIC ID with vector 47
    ioapic_route_irq(16, bsp_lapic_id, 48, bsp_flags);     // Route IRQ 16 to current LAPIC ID with vector 48
    ioapic_route_irq(17, bsp_lapic_id, 49, bsp_flags);     // Route IRQ 17 to current LAPIC ID with vector 49
    ioapic_route_irq(18, bsp_lapic_id, 50, bsp_flags);     // Route IRQ 18 to current LAPIC ID with vector 50

    ioapic_route_irq(140, bsp_lapic_id, 172, bsp_flags);   // Route IRQ 140 to current LAPIC ID with vector 172
    ioapic_route_irq(141, bsp_lapic_id, 173, bsp_flags);   // Route IRQ 141 to current LAPIC ID with vector 173
    ioapic_route_irq(142, bsp_lapic_id, 174, bsp_flags);   // Route IRQ 142 to current LAPIC ID with vector 174

    printf(" [-] IOAPIC IRQs routed to BSP core %d\n", bsp_lapic_id);

    if(has_fpu()){
        enable_fpu_and_sse();
        printf(" [-] FPU and SSE enabled for BSP core %d\n", bsp_lapic_id);
    }
    

    cpu_datas[bsp_lapic_id].lapic_id = bsp_lapic_id; // Set the LAPIC ID for the bootstrap core
    cpu_datas[bsp_lapic_id].smp_info = smp_response->cpus[bsp_lapic_id]; // Set the SMP info for the bootstrap core 
    cpu_datas[bsp_lapic_id].is_online = 1; // Mark the bootstrap core as online
    cpu_datas[bsp_lapic_id].cpu_stack = read_rsp(); // Set the stack pointer for the bootstrap core

    asm volatile("sti"); // Enable interrupts

    printf("[Info] Bootstrap CPU %d initialized...\n\n", bsp_lapic_id);
}


void target_cpu_task(struct limine_smp_info *smp_info) {

    if(smp_info == NULL){
        printf("[Error] SMP INFO is null!\n");
        return;
    }

    uint32_t core_id = smp_info->lapic_id;

    cpu_datas[core_id].lapic_id = core_id;
    cpu_datas[core_id].smp_info = smp_info;

    disable_pic();

    // setting cr3 for this core
    init_core_paging(core_id);
    
    // Initialize the stack for this core
    uint64_t cpu_stack = (uint64_t)kmalloc_a(STACK_SIZE, 1); // Allocate stack for this core
    if(cpu_stack == 0) {
        printf("[Error] Failed to allocate stack for CPU %d\n", core_id);
        return;
    }
    uint64_t cpu_stack_top = cpu_stack + STACK_SIZE; // Set the stack pointer to the top of the allocated stack
    cpu_datas[core_id].cpu_stack = cpu_stack_top;    // Set the stack pointer to the top of the allocated stack
    set_rsp(cpu_stack_top);                          // Set the stack pointer for this core

    // Initialize GDT and TSS for this core
    init_gdt_tss_in_cpu(core_id);

    // Initialize interrupts for this core
    init_core_interrupt(core_id);

    // Initialize APIC for this 
    init_apic_interrupt();

    // if(has_fpu()){
    //     enable_fpu_and_sse();
    // }

    init_apic_timer(100);

    cpu_datas[core_id].is_online = 1; // Mark this core as online
    

    printf(" [-] CPU %d (LAPIC ID: %x) is online\n", core_id, core_id);

    while(true){
        if(core_id == 0 && cpu_datas[core_id].is_online == 1){
            // This is the bootstrap core (BSP)
            printf(" Hello from CPU %d (BSP)\n", core_id);
            for(volatile int i = 0; i < 2000000; i++); // Wait for a while

        } else if (core_id == 1 && cpu_datas[core_id].is_online == 1){
            printf(" Hello from CPU %d (AP)\n", core_id);
            for(volatile int i = 0; i < 1900000; i++); // Wait for a while

        } else if (core_id == 2 && cpu_datas[core_id].is_online == 1){
            printf(" Hello from CPU %d (AP)\n", core_id);
            for(volatile int i = 0; i < 1800000; i++); // Wait for a while

        } else if (core_id == 3 && cpu_datas[core_id].is_online == 1){
            printf(" Hello from CPU %d (AP)\n", core_id);
            for(volatile int i = 0; i < 1700000; i++); // Wait for a while
        } else{
            continue;
        }
    }

    asm volatile("sti"); // Enable interrupts

    // Halt the AP as we have nothing else to do currently
    for (;;) {
        asm volatile("hlt");
    }
}


void start_ap_cpu_cores() {

    if (smp_response == NULL) {
        printf("[Error] SMP response is null!\n");
        return;
    }

    uint32_t start_id = 1;
    uint32_t end_id = smp_response->cpu_count; // Get the number of CPUs from the SMP response
    
    for (uint32_t core = start_id; core < end_id; core++) {

        struct limine_smp_info *smp_info = smp_response->cpus[core];

        // passing sm_info as argument which will accept as input by target_cpu_task
        smp_info->extra_argument = (uint64_t)smp_info;

        // Set function to execute on the AP
        smp_info->goto_address = (limine_goto_address)target_cpu_task;

        for(volatile int i = 0; i < 1000000; i++); // Wait for a while
    }
}



// Initializing all CPU cores
void init_all_cpu_cores() {

    // Initialize the bootstrap core
    start_bootstrap_cpu_core();

    // Initialize the application cores
    start_ap_cpu_cores();

    // Wait for all cores to be online
    while (1) {
        int all_online = 1;
        for (size_t i = 0; i < smp_response->cpu_count; i++) {
            if (!cpu_datas[i].is_online) {
                all_online = 0;
                break;
            }
            printf(" [-] CPU %d is online\n", i);
        }
        if (all_online) {
            break;
        }
    }
    asm volatile("sti");

    printf("[Info] All CPU cores initialized and online.\n");
}


void switch_to_core(uint32_t target_lapic_id) {
    if (smp_response == NULL) return;

    for (size_t i = 0; i < smp_response->cpu_count; i++) {
        struct limine_smp_info *smp_info = smp_response->cpus[i];
        if (smp_info->lapic_id == target_lapic_id) {
            // Send IPI to the target core
            lapic_send_ipi(target_lapic_id, 0x40); // 0x20 is the vector
            break;
        }
    }

    printf("[Info] Successfully Switched  into CPU %d.\n", target_lapic_id);
}
Here I am getting following output without interrupt stoped:

Code: Select all

[Info] Starting bootstrap CPU core...
 [-] PIC Disabled.
 [-] ACPI 1.0 is signature and checksum validated
[Error] FADT not found! ACPI status unknown.
 [-] Successfully ACPI Enabled
 [-] TSC Timer initialized with CPU Frequency 168125 Hz
 [-] GDT & TSS initialized.
 [-] Successfully Bootstrap CPU 0 Interrupt Initialized.
 [-] Interrupt Based System Call initialized!
 [-] APIC Timer initialized with 100 ms interval in CPU: 0.
 [-] Successfully KEYBOARD initialized.
 [-] IOAPIC IRQs routed to BSP core 0
 [-] FPU and SSE enabled for BSP core 0
[Info] Bootstrap CPU 0 initialized...
The QEmu log is showing following output:

Code: Select all

CPU Reset (CPU 3)
EAX=000f49ee EBX=00000000 ECX=000002ff EDX=00000003
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00006c5c
EIP=000fd0ac EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00cf9b00 DPL=0 CS32 [-RA]
SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000f6180 00000037
IDT=     000f61be 00000000
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=00000000 CCD=00000004 CCO=EFLAGS
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=0000000000000000 0000000000000000 XMM01=0000000000000000 0000000000000000
XMM02=0000000000000000 0000000000000000 XMM03=0000000000000000 0000000000000000
XMM04=0000000000000000 0000000000000000 XMM05=0000000000000000 0000000000000000
XMM06=0000000000000000 0000000000000000 XMM07=0000000000000000 0000000000000000
     0: v=30 e=0000 i=0 cpl=0 IP=0008:ffffffff80003ce3 pc=ffffffff80003ce3 SP=0010:ffff80007ff41e50 env->regs[R_EAX]=0000000000000060
RAX=0000000000000060 RBX=0000000000000000 RCX=0000000000000000 RDX=00000000000003fd
RSI=00000000000002b0 RDI=00000000000003fd RBP=ffff80007ff41e60 RSP=ffff80007ff41e50
R8 =00000000000000ee R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff80003ce3 RFL=00000296 [--S-AP-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 0fffffff 00a09a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
DS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
FS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
GS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0028 ffffffff805a8fc0 00000068 00008900 DPL=0 TSS64-avl
GDT=     ffffffff805a8f60 00000037
IDT=     ffffffff805aa760 00000fff
CR0=80010011 CR2=0000000000000000 CR3=000000007ff31000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000094 CCD=ffff80007ff41e28 CCO=EFLAGS
EFER=0000000000000d00
     1: v=30 e=0000 i=0 cpl=0 IP=0008:ffffffff80003d00 pc=ffffffff80003d00 SP=0010:ffff80007ff41e50 env->regs[R_EAX]=000000000000005b
RAX=000000000000005b RBX=0000000000000000 RCX=0000000000000000 RDX=00000000000003f8
RSI=000000000000005b RDI=00000000000003f8 RBP=ffff80007ff41e60 RSP=ffff80007ff41e50
R8 =00000000000000ee R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff80003d00 RFL=00000292 [--S-A--] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 0fffffff 00a09a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
DS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
FS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
GS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0028 ffffffff805a8fc0 00000068 00008900 DPL=0 TSS64-avl
GDT=     ffffffff805a8f60 00000037
IDT=     ffffffff805aa760 00000fff
CR0=80010011 CR2=0000000000000000 CR3=000000007ff31000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000090 CCD=ffff80007ff41e38 CCO=EFLAGS
EFER=0000000000000d00
     2: v=30 e=0000 i=0 cpl=0 IP=0008:ffffffff80003d00 pc=ffffffff80003d00 SP=0010:ffff80007ff41e50 env->regs[R_EAX]=0000000000000020
RAX=0000000000000020 RBX=0000000000000000 RCX=0000000000000000 RDX=00000000000003f8
RSI=0000000000000020 RDI=00000000000003f8 RBP=ffff80007ff41e60 RSP=ffff80007ff41e50
R8 =00000000000000ee R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff80003d00 RFL=00000292 [--S-A--] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 0fffffff 00a09a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
DS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
FS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
GS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0028 ffffffff805a8fc0 00000068 00008900 DPL=0 TSS64-avl
GDT=     ffffffff805a8f60 00000037
IDT=     ffffffff805aa760 00000fff
CR0=80010011 CR2=0000000000000000 CR3=000000007ff31000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000090 CCD=ffff80007ff41e38 CCO=EFLAGS
EFER=0000000000000d00
     3: v=30 e=0000 i=0 cpl=0 IP=0008:ffffffff80003d00 pc=ffffffff80003d00 SP=0010:ffff80007ff41e50 env->regs[R_EAX]=0000000000000054
RAX=0000000000000054 RBX=0000000000000000 RCX=0000000000000000 RDX=00000000000003f8
RSI=0000000000000054 RDI=00000000000003f8 RBP=ffff80007ff41e60 RSP=ffff80007ff41e50
R8 =00000000000000ee R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff80003d00 RFL=00000292 [--S-A--] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 0fffffff 00a09a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
DS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
FS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
GS =0010 0000000000000000 0fffffff 00a09300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0028 ffffffff805a8fc0 00000068 00008900 DPL=0 TSS64-avl
GDT=     ffffffff805a8f60 00000037
IDT=     ffffffff805aa760 00000fff
CR0=80010011 CR2=0000000000000000 CR3=000000007ff31000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000090 CCD=ffff80007ff41e38 CCO=EFLAGS
EFER=0000000000000d00
The above qemu log have v=30 which is due to apic interrupt but without not showing apic_handler messages and actually qemu is not receiving any interrupt request(keyboard) and not showing coerresponding irq_handler function result.
Where my maiden GitHub Repo .

How could I resolve this issue?
Top
rdos
Member
Member
Posts: 3357
Joined: Wed Oct 01, 2008 1:55 pm

Re: What is the reason apic interrupt is not working in application processor?

Post by rdos »

I have no idea why you initialize the hardware on both processor cores. You should only initialize hardware on one core, typically on the BSP. If you want hardware interrupts to your ap processor, you must set up the IOAPIC with the correct processor core (APIC id) for this source.
Top
Post Reply

Return to “OS Development”