Use ioeventfd instead of VMExit for syscalls

[MatiasVara]

This issue shall replace the current mechanism of trapping syscalls that is based on a heavy VMExit with ioeventfd. By using this mechanism, the VMM will be notified asynchronously that the app requires a syscall. This happens when the app writes to a specific mmio register. This is the mechanism used in vhost and for the virtqueue notification. This seems to perform better than user-space VMExits. The new mechanism would be:

1. app guest writes to special mmio reg
2. Event is triggered in the thread that processes eventfd
3. The thread processes the request and tells the app guest that is done
4. app guest resumes execution. Here I am sure how the VMM should tell the guest that the syscall is done.

• Implement eventfd in fpc linux unit (https://fpc-pascal.freepascal.narkive.com/w3YSd13n/using-eventfd)
• Add struct kvm_ioeventfd
• Implement int ioeventfd__init(struct kvm *kvm)
• Implement static int ioeventfd__start(void)

[1] see virtio_mmio_init_ioeventfd() at https://github.com/clearlinux/kvmtool/blob/b5891a4337eb6744c8ac22cc02df3257961ae23e/virtio/mmio.c
[2] see ioeventfd__add_event() at https://github.com/clearlinux/kvmtool/blob/master/ioeventfd.c
[3] http://blog.allenx.org/2015/07/05/kvm-irqfd-and-ioeventfd

[MatiasVara]

The following structure needs to be defined:

struct kvm_ioeventfd {
	__u64 datamatch;
	__u64 addr;        /* legal pio/mmio address */
	__u32 len;         /* 1, 2, 4, or 8 bytes; or 0 to ignore length */
	__s32 fd;
	__u32 flags;
	__u8  pad[36];
};

[MatiasVara]

The event is configured as follows:

kvm_ioevent = (struct kvm_ioeventfd) {
		.addr		= vmmio->addr + VIRTIO_MMIO_QUEUE_NOTIFY,
		.len		=sizeof(u32),
		.datamatch	= value,
		.fd		=   eventfd(0, 0),
		.flags		= KVM_IOEVENTFD_FLAG_DATAMATCH,
};

Then, the event is registered as follows:

r = ioctl(ioevent->fn_kvm->vm_fd, KVM_IOEVENTFD, &kvm_ioevent);
if (r) {
	r = -errno;
	goto cleanup;
}

[MatiasVara]

When pooling in user-space, it does the following:

epoll_event = (struct epoll_event) {
		.events		= EPOLLIN,
		.data.ptr	= new_ioevent,
};
r = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, event, &epoll_event);
if (r) {
	r = -errno;
	goto cleanup;
}
list_add_tail(&new_ioevent->list, &used_ioevents);

[MatiasVara]

We need to implement something like in int ioeventfd__init(struct kvm *kvm) and static int ioeventfd__start(void).

[MatiasVara]

A thread is executing a epoll_wait() until a event is received. When an event is recevied, the callback is executed:

static void virtio_mmio_ioevent_callback(struct kvm *kvm, void *param)
{
	struct virtio_mmio_ioevent_param *ioeventfd = param;
	struct virtio_mmio *vmmio = ioeventfd->vdev->virtio;

	ioeventfd->vdev->ops->notify_vq(kvm, vmmio->dev, ioeventfd->vq);
}

[MatiasVara]

I need to figure out how to tell the guest that the syscall is done since this mechanism is async.

[MatiasVara]

Once way may be to use a virtio-device to exchange data to the host.