Qwen3.8 09 · RoPE and Position

No rotary-position change improved the benchmark. The best validation result remained the RTX 3090 baseline of 1.248862 bits per byte; the only alternative that completed evaluation was worse, while the layer-wise variant never produced a valid measurement.

Measured improvement0.0% better

Started: 1.2489
Best: 1.2489

Best-so-far result across 8 recorded experiments. Lower text prediction loss is better. Original research record

Raising the RoPE base hurt validation

Changing the RoPE base from 10,000 to 500,000 increased validation loss from 1.248862 to 1.258732. Because lower is better, this was a regression of 0.009870 bits per byte and was correctly rejected.

At this fixed sequence length, the much larger base did not improve the model’s use of positional information. The practical conclusion is narrow but clear: retain the 10,000 base rather than adopting 500,000 for this benchmark.

The layer-wise idea remains untested

The other hypothesis assigned progressively larger bases to deeper layers, using 10000 × 1.05^layer. In principle, this was intended to give shallow layers finer local positional resolution and deeper layers lower-frequency, longer-range information.

However, all six attempts failed while constructing the model, when the separate per-layer cosine tables were being stacked and registered. None reached validation, so there is no evidence that the layer-wise schedule helps or hurts prediction loss. Repeating essentially the same failing implementation consumed most of the remaining experiments without adding measurements.

Before spending another five-minute GPU evaluation on this idea, the per-layer rotary tables should be tested with a lightweight model-construction and forward-pass check. Only after that check succeeds would the hypothesis receive a fair comparison.

What to keep and what is still unknown

Keep the baseline rotary configuration with base 10,000. It is the only accepted configuration and remains the measured best at 1.248862 bits per byte.

This run does not establish whether partial rotary dimensions, alternative frequency construction, rotary application choices, or working layer-wise variants can improve the result: none of those choices produced a completed comparison. With only two valid measurements among eight experiments, the broader RoPE search space remains largely unexplored.

Bio

Hello, I’m İbrahim Sait Akarçeşme. I was born on March 8, 2005, in Lüleburgaz, a town located in the European part of Türkiye. Both of my parents are literature teachers. As a child, I struggled to adapt to school and changed schools frequently. My parents’ profession also required our family to move often, which made that sense of instability even more pronounced.

The earliest memories I can clearly visualize begin in Bursa, around the time I started primary school. I was a child who loved playing football, although I was never particularly talented at it. I disliked school and hated doing homework. I have an older sister and a younger brother.

My primary school years in Bursa were shaped by constant transitions. I first spent a year at one of the city’s most modern private schools. Then I transferred to a public school. During our final year in Bursa, I attended the public school for the first semester before returning, in the second semester, to the private school where I had originally begun my education.

During those years, my father decided to take steps toward becoming the mayor of Çamoluk, our family’s hometown, in order to repay what he described as a debt of gratitude to the place he came from. I was too young to fully understand what was happening at the time. However, after he was elected mayor of one of Türkiye’s least socioeconomically developed districts, our family moved there.

I went from studying at one of the country’s most modern private schools to attending what could reasonably be described as a village school in one of Türkiye’s smallest districts. I completed the rest of primary school and continued there until the end of middle school. It was an unusual period of my life, filled with experiences that others might describe as traumatic. Still, I see those years as a period that led to some of the most important changes in my character and perspective.

After spending four years in that small district, we moved to Sakarya. That was also the year I began preparing for the high school entrance exam. I was the kind of student who listened carefully in class but rarely studied afterward and almost never completed homework. Under ordinary circumstances, this might have made me a fairly typical student. But when you have an exceptionally intelligent and successful older sister, and both of your parents are educators, the pressure can become somewhat greater.

The summer following the entrance exam became one of the most important turning points in my life. My score placed me directly between two possibilities. I would either narrowly qualify for one of the schools I genuinely wanted, or I would end up at the school I had placed at the bottom of my list as a worst-case option. I do not remember ever praying as sincerely as I did during that period.

Yet I believe there is good in everything. In the end, I was admitted to the school I had not wanted to attend. That is where the high school years that shaped my character, vision, and ambitions began.

In high school, I was fortunate to become part of a remarkable group of friends who pushed me to grow in ways I had never imagined. Many of them later went on to study medicine, law, and engineering. With two of those friends in particular, despite being only fifteen or sixteen years old, we decided to build a startup rather than allow the limits and expectations of the people around us to define what was possible.

As our first step, we decided to develop a tourism project for Türkiye’s largest technology festival, competing in a newly introduced and highly competitive category. That project was called Mobil Çelebi.

Although we were still in high school and spent much of that period confined to our homes during the pandemic, I would probably describe those years as some of the happiest of my life. Imagine teenagers waking up at five in the morning, drinking coffee they did not even enjoy, and meeting on Discord to watch programming courses together.

It was difficult, but the three of us managed to maintain that level of discipline. We achieved something many people around us considered nearly impossible and earned one of the first major victories of our lives at Türkiye’s largest technology festival.

I still remember one of the judges telling us, “Even if you had competed against university students, you would still have won.” What made Mobil Çelebi more than a competition project, however, was that we continued developing it after the event was over. Even years later, we never completely lost faith in its purpose or its potential.

Had we been living in Europe or the United States, I believe we might have been able to preserve the momentum we built during high school, and perhaps today we would be speaking very differently about ourselves and our work. But the realities of Türkiye’s education system and the circumstances of that period pushed us toward preparing for the national university entrance exam. As a result, we were forced to step away from our projects, at least temporarily.

Once again, there was good in what followed. After the exam, all three of us were accepted into engineering programs. I decided to leave Türkiye and move to Poland, while my two friends chose to continue their university education at home.

When I first arrived in Poland, I had little idea what was waiting for me. During my first year, I completed a Polish-language preparatory program and experienced living in a student dormitory for the first time. After that, my formal education in Computer Science began.

Despite the temporary distance created by living in different countries, those two friends from high school continue to build meaningful things through their startups and their extraordinary potential. I sincerely wish them every success.

I see my years in Poland as a period in which I made several radical decisions about my inner life. Here, I met people whom I can describe as closer than brothers. One might expect that, after leaving my family and living abroad, I would abandon the beliefs and values I had grown up with in the name of freedom. Strangely, the opposite happened. I embraced them more deeply. Today, I have no doubt about my faith or about the responsibilities it places upon me.

If I once had a tendency to prioritize personal gain or self-interest, I feel that I have gradually lost it since coming to Poland. Money, property, cars, career, and travel no longer give me anything close to the sense of meaning I find in the hope of contributing to struggles for peace and freedom.

I could say that my philosophy of life was shaped in Poland. It was here that I first learned how to travel alone, meet my own needs, and understand what it truly means to be by yourself. For that reason, if someone asked where I feel I belong, I would probably answer: Warsaw.

Although Poland contributed greatly to my inner development, leaving behind the extraordinary group of friends I had in Türkiye distanced me, for a while, from the vision and ambition I once carried. That continued until the beginning of 2026, when I began to recover that momentum.

Around that time, I had the opportunity to meet some of the most remarkable people I have ever known. Since then, I have been doing everything I can to develop myself as much as possible. I believe the coming years will be exceptionally dynamic, particularly because of the transformations taking place in technology and artificial intelligence, as well as the decisions and responsibilities those changes may require from us.

I have spoken at length about my past. Now, let me say something about the future.

I was never raised as a child whose dreams or ambitions were meant to be limited. I have the kind of father who, if I told him, “I want to fly,” would not simply encourage me to fly, but would tell me to discover the stars as well. At the same time, I have a mother who makes sure I never lose touch with reality. If I told her, “I want to fly,” she would probably say, “Be careful not to catch a cold.”

That is why my dreams reach beyond the stars, while remaining as real as the wind touching our skin and awakening even the smallest of our senses.

I have no doubt that the world has enough resources. I have no doubt that good people exist. And I believe those people can help create a world built on peace, justice, and a form of prosperity in which no one is left in poverty.

My only hope is that I can support them and play even a small part in what I consider one of humanity’s most valuable, and perhaps most sacred, responsibilities.

My name is İbrahim Sait Akarçeşme.

I hope you remember it.