Creating a real time linux threads from bash and linux scheduling

I am new to linux kernel development. I was trying to learn thread creation and synchronization. My end goal is to create two threads and the two threads use a shared resource guarded by a semaphore.

The code is

#include<stdio.h>
#include<string.h>
#include<pthread.h>
#include<stdlib.h>
#include<unistd.h>
#include<semaphore.h>

sem_t sema_obj;

pthread_t tid[2];

int shared_val = 0;

void* doSomeThing(void *arg)
{
    unsigned long i = 0;
    pthread_t id = pthread_self();

    for(i=0;i<5;i++){
    printf("\n going to wait %x\n",(unsigned int)id);
    sem_wait(&sema_obj);
    shared_val++;
    sleep(1);
    printf("\n %d The value of shared_val is %d in thread %x \n",(int)i, shared_val, (unsigned int)id);
    sem_post(&sema_obj);
    printf("\n gave up sem %x\n",(unsigned int)id);
    }

    for(i=0; i<(0xFFFFFFFF);i++);

    return NULL;
}

int main(void)
{
    int i = 0;
    int err;
    sem_init(&sema_obj, 0, 1);
    while(i < 2)
    {
    pthread_attr_t attr;
    struct sched_param param;

    pthread_attr_init(&attr);
    pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
    param.sched_priority = 50;
    pthread_attr_setschedparam(&attr, &param);
    //sched_setscheduler(current, SCHED_FIFO, &param);
        err = pthread_create(&(tid[i]), &attr, &doSomeThing, NULL);
    //err = pthread_create(&(tid[i]), NULL, &doSomeThing, NULL);
        if (err != 0)
            printf("\ncan't create thread :[%s]", strerror(err));
        else
            printf("\n Thread created successfully 0x%X \n",(unsigned int)tid[i]);

        i++;
    }
    enter code here
    sleep(60);
    return 0;
}

I am compiling it with

gcc -o threads_op -pthread -lrt threads.c

After I run the code I see:

 Thread created successfully 0xB75CBB40 

 going to wait b75cbb40

 Thread created successfully 0xB6DCAB40 

 going to wait b6dcab40

 0 The value of shared_val is 1 in thread b75cbb40 

 gave up sem b75cbb40

 going to wait b75cbb40

 1 The value of shared_val is 2 in thread b75cbb40 

 gave up sem b75cbb40

 going to wait b75cbb40

 2 The value of shared_val is 3 in thread b75cbb40 

 gave up sem b75cbb40

 going to wait b75cbb40

 3 The value of shared_val is 4 in thread b75cbb40 

 gave up sem b75cbb40

 going to wait b75cbb40

 4 The value of shared_val is 5 in thread b75cbb40 

 gave up sem b75cbb40

 0 The value of shared_val is 6 in thread b6dcab40 

 gave up sem b6dcab40

 going to wait b6dcab40

 1 The value of shared_val is 7 in thread b6dcab40 

 gave up sem b6dcab40

 going to wait b6dcab40

 2 The value of shared_val is 8 in thread b6dcab40 

 gave up sem b6dcab40

 going to wait b6dcab40

 3 The value of shared_val is 9 in thread b6dcab40 

 gave up sem b6dcab40

 going to wait b6dcab40

 4 The value of shared_val is 10 in thread b6dcab40 

 gave up sem b6dcab40

The RTPRIO is '-' for this process, S 1000 4551 4338 - 00:00:00 threads_op

Had the following questions,

1) Why is the thread not real time and taking the priority I am setting ?

2) Why is the semaphore not getting locked alternately by the two threads and update the share variable ? The two threads have same priority.

If you know a step by step hands on tutorial to understand linux kernel topics, please share.

A few things.

First, I hate to be a nit about things, but you're not doing linux kernel development. You're writing a multithreaded application that runs on linux. Thus, the linux-kernel tag is inappropriate. Kernel development is about writing code that is built into the kernel, such as schedulers, device drivers, filesystem drivers, etc.

You've got a sleep inside your critical section (e.g. between sem_wait and sem_post). You're experiencing "thread starvation". See my answer here: Thread Synchronization C++

Since you're using pthreads, you'd probably be better off with pthread_mutex_lock/pthread_mutex_unlock.

You're also using printfs. They disrupt the timing, even for debug.

You're trying to do SCHED_FIFO but not checking the return code. Only root can set some RT schedulers and/or high[er] priorities. Likewise, for scheduling priority.

Wow! Priority 50 for SCHED_FIFO? If you actually got it, anything higher than 11 will schedule you at a higher priority than some internal kernel threads. You would probably lock up the system solid. [*]

At the stage you're at, before you go "all in" [with RT scheduler, etc.], work with the normal scheduler at standard priority. You'll be amazed at how well it will perform. Debug your thread synchronization code/logic first before cranking up the voltage.

In reality, for a smooth highly responsive RT app that is deterministic with low latency, I think you'll need a bit more planning about how you'll synchronize, what queues between threads, types of locking (e.g. RCU, trylocks, timeouts on lock attempts, etc.) How many threads will you need? What will each one do? Do you have pseudocode for each? What type of thread environment (e.g. consumer/producer, master/slave/worker). Are your threads going to do "work stealing"? Etc, etc, etc. Will you use IPC messaging, shared memory? How much data? What bandwidth for the data between threads?

[*] The reason I know this is that I was a developer on a realtime system that had 50 threads, 8 cores, and constant interaction with multiple device drivers and FPGAs and massive amounts of DMA [so much so, we had to measure it with a PCIe bus analyzer to be sure the bus had enough bandwidth]. It's now a shipping product.