CS 590/690 SEMINARS
Fourier Vision Transformer with Efficient Token Mixing for Image Classification
Barış Bilgin Şenol
Master Student
(Supervisor: Asst.Prof.Özgür Salih Öğüz)
Computer Engineering Department
Bilkent University
Abstract: The traditional success of the Transformer model is often credited to its self-attention mechanism for mixing tokens. However, recent research indicates that alternative mixing techniques can also achieve effective results in various applications, emphasizing the significance of the overall architecture rather than just the token mixer. In response, we present the Fourier Vision Transformer (FoViT), which integrates the Discrete Fourier Transform to enhance the computational efficiency of the Transformer model. This novel approach harnesses the unique properties of the Discrete Fourier Transform, employing frequency representations for efficient token mixing and modeling long- range feature interactions. Extensive testing on image classification tasks shows that the Fourier Vision Transformer surpasses the original Vision Transformer in throughput and memory efficiency, while still delivering comparable accuracy.
DATE: April 01, Monday @ 13:30 Place: EA 502
Modular Reinforcement Learning of Teamed Adversarial Agents in Turn Based Tactics Games
Kaan Ünlü
Ph.D. Student
(Supervisor: Asst.Prof.Özgür Salih Öğüz)
Computer Engineering Department
Bilkent University
Abstract: Turn Based Tactics Games are a subgenre of strategy games revolving usually around multi-map, multi-goal and multi-unit based, goal oriented frameworks; in many ways multidimensional expansions of chess, complexity-wise. As one of the earliest and most famous pioneers of Reinforcement Learning, AlphaZero has paved the way for Game RL in academia with chess, but many quirks of RL, especially its inflexibility in dynamically working in different maps is yet unsolved. As such, first we will present previous literature and our work with Deep Reinforcement Learning in our custom environment. After that, we will discuss the limitations in agent collaboration portrayed by our work, and look into further methodology through literature in state abstraction and curriculum learning as potential solutions.
DATE: April 01, Monday @ 13:50 Place: EA 502
Hyperparameter aware partitioning for efficient GNN training
Emre Erdal
Master Student
(Supervisor: Prof.Dr.Cevdet Aykanat)
Computer Engineering Department
Bilkent University
Abstract: In this project, we explore how to better balance the computational load in training multilayer Graph Neural Networks (GNNs). The need for this comes from the fact that in GNNs, the amount of computation needed can change a lot between layers due to different settings, or “hyperparameters,” making some parts of the training process heavier on the system than others. We will investigate and implement different combinatorial models and methods to tackle this problem in order to scale parallel training of multilayer GNNs on distributed-memory systems.
DATE: April 01, Monday @ 14:10 Place: EA 502
Copy number estimation using Counting Bloom Filters in de novo assembled genomes
Klea Zambaku
Master Student
(Supervisor: Assoc.Prof.Can Alkan)
Computer Engineering Department
Bilkent University
Abstract: Genomes of complex organisms contain a large amount of repetitive sequences. These repeats provide elasticity to genomes, which in turn guide evolutionary processes; however, some are also associated with several diseases either directly or indirectly through facilitating genome rearrangements. On the other hand, repeats also contribute to misassemblies due to the ambiguities they create in paths in genome assembly graphs. To detect and resolve such ambiguities, efficiently estimating the copy numbers of repeats is of interest. Here, we propose to estimate copy numbers using Bloom Filters and Counting Bloom Filters in both de novo genome assemblies and whole genome sequences to improve both run time and memory footprint. Our method implements spaced seeds to generate k-mers from genome assemblies, which are used to populate a Bloom Filter. We then check for the presence of k-mers in long reads and populate a Counting Bloom Filter if they are found in the first Bloom Filter. Next, we translate these results into segments with estimated copy numbers. In our experiments, we used long and short reads from several organisms and compared our results to existing short-read copy number prediction algorithms. We also validated our results by comparing them to MegaBlast and RepeatMasker as the ground truth. Our method will be helpful in the future as a pipeline that resolves unresolved misassemblies built by de novo assembly algorithms and is efficient in estimating copy numbers from long-read sequencing datasets.
DATE: April 01, Monday @ 14:30 Place: EA 502