I was doing some thinking today. I once built a LEGO robot car that used a Raspberry Pi as the controlling computer, which was interfaced to an Arduino that operated the drive train and a servo that operated the rack and pinion steering. It had a web interface for controlling the car that also supported speech control that was driven by cloud AI. I used a built in browser feature now available in most browsers that will translate voice into text in real-time. I wrote some Javascript on the web page that would then send the text to an AI engine (I used wit.ai which is owned by Facebook/Meta). I have setup a model with intents that would choose from a set of commands and parameters, which would then be returned to the web page. The web page would then map these to Ajax calls to the web server running on the car which would execute the command. And the browser would speak a response provided by the car.
I could say things like "Robot, go forward and turn left." or "Drive forward and hang a left." In both cases the AI knew what I wanted and the car would drive forward and turn left at the same time. I did not have to train my voice or the specific phrases. It saw two "intents", one for forward motion and one for steering left. It had a bunch of words mapped to each of these intents. As long as the intent was found, it could command the car.
So this got me thinking. I wonder if it would be possible to build a mod for DCS that understands where all the friendlies are and where all the bad guys are (like LotATC does), is able to listen in on an SRS channel that, converting what it hears to text, then using AI to interpret the context and respond to the player with things like picture, vectors, etc, and convert the response to audio using text to speech and then say the response on the same radio channel. One could build up a rudimentary automated GCI. I think Hoggit might have something like this already but I am not sure. Maybe worth some additional research. Thoughts?
BTW, here is a video of my robot Lego car being commanded by voice...
AI GCI
Moderator: RLG MGMT Team
Re: AI GCI
There is already a mod called DCS-STTS that allows you to generate text to speech and have it transmitted on SRS using any radio frequency you want. The mod source code is on GitHub so I should be able to study exactly how they are doing that. I am going to attempt to see if I can reverse engineer how LotATC is getting the positions of all of the units. If I can't figure that out, I may have to sniff the network traffice between the LotATC server and the LotATC client and make my own client for my own nefarious purposes. ;)
Re: AI GCI
I looked at what DCS-STTS is doing, and it's basically shelling out to an .EXE that is included with SRS that you pass in a set of command line arguments to, and it automatically generates the audio for you and sends it out on SRS on the frequency you specified. The source code is fully available for SRS, including the the text to audio transmit utility that DCS-STTS is using. For example: you can could execute:
DCS-SR-ExternalAudio.exe --text="I want a male voice with a German accent" --freqs=251.0 --modulations=AM --coalition=1 --gender=male --locale=de-DE --googleCredentials="C:\folder\credentials.json"
And this program will generate a German-sounding voice that says "I want a male voice with a German accent" and it will transmit it on 251.0 AM in SRS. Pretty cool stuff.
DCS-SR-ExternalAudio.exe --text="I want a male voice with a German accent" --freqs=251.0 --modulations=AM --coalition=1 --gender=male --locale=de-DE --googleCredentials="C:\folder\credentials.json"
And this program will generate a German-sounding voice that says "I want a male voice with a German accent" and it will transmit it on 251.0 AM in SRS. Pretty cool stuff.
Re: AI GCI
Also the same utility can be used to transmit .mp3 and .ogg files to SRS on any frequency. That could also be very useful. So basically I need to figure out how to get the data out of DCS, and I need to figure out how to listen in on an SRS channel and send sound bytes to get translated to text, process them with AI, and then generate the and transmit the audio output. The listening part and the getting the data from DCS or LotATC are the hard parts. The rest of the equation is already solved in my mind.
Re: AI GCI
So I played around a little with the C#. Net native speech recognition class and it looks like it could be usable. It is able to bind to an audio device to listen to. There is also the possibility of using Google Cloud SDK speech recognition with c#. Net but that might come with some limits and cost if those limits are exceeded. Anyway, I think this project could be doable. I may work on a proof of concept to see how it might work. I believe I can get it to ignore anything that is said that does not include its callsign. For example, "Magic, Striker 1 1 requests bogey dope." would be processed, but somebody saying "This bud is for you" would be ignored. It would pick up on "magic" anywhere in the sentence, though, and attempt to figure out the intent. Basically I can create a model with different triggers so that things said that don't include at least one of those triggers gets ignored.
Re: AI GCI
I have been thinking about this all night and how I would implement things. I would make the software track every player and have it maintain a state for each player, which the player can change. For example:
"Magic, Dodge 1-1 checking in as fragged". Now the AI may not know what your mission is, but it now knows you have checked in and we cna talk to you and give you advisories.
"Magic, Dodge 1-1 is Winchester, R T B"
Now the AI knows you are heading back and cannot engage enemy.
"Magic, Dodge 1-1 request vectors to mother"
Now AI knows you want to return to the ship, and could give you initial vectors and periodically update you, maybe even occasionally routing you out of harms way.
Same goes for returning to an airport. I doubt the speech recognition will understand the names of the airports that well (and we all know I cannot pronounce them properly to save my life) but we could give the ones we operate out of codenames that the AI will recognize.
The AI won't be aware of mission objectives. It could be, but that would require serious scripting and nodding of the missions themselves - which is a whole different animal. For now, I plan to build a POC that has limited functionality but will be able to give you nearest contacts, vectors to airports and ships, vectors to friendlies, tankers, etc.
If that all works, I can see about expanding the capabilities over time. It would be cool to have it handle ATC for example. Maybe way down the road it could hook in with a mission and generate dynamic CAS missions.
"Magic, Dodge 1-1 checking in as fragged". Now the AI may not know what your mission is, but it now knows you have checked in and we cna talk to you and give you advisories.
"Magic, Dodge 1-1 is Winchester, R T B"
Now the AI knows you are heading back and cannot engage enemy.
"Magic, Dodge 1-1 request vectors to mother"
Now AI knows you want to return to the ship, and could give you initial vectors and periodically update you, maybe even occasionally routing you out of harms way.
Same goes for returning to an airport. I doubt the speech recognition will understand the names of the airports that well (and we all know I cannot pronounce them properly to save my life) but we could give the ones we operate out of codenames that the AI will recognize.
The AI won't be aware of mission objectives. It could be, but that would require serious scripting and nodding of the missions themselves - which is a whole different animal. For now, I plan to build a POC that has limited functionality but will be able to give you nearest contacts, vectors to airports and ships, vectors to friendlies, tankers, etc.
If that all works, I can see about expanding the capabilities over time. It would be cool to have it handle ATC for example. Maybe way down the road it could hook in with a mission and generate dynamic CAS missions.
Re: AI GCI
Hey XPen,
I think what you're intending to build here has indeed been done. The program is called OverlordBot 2.3.0, and it serves as an improved AWACS for DCS. It works with SRS, and the videos on youtube suggests it works fairly well. The problem with it is that after a certain amount of usage by players, the server owners are charged a fee. I think it takes quite a lot of request to the bot by players before the company starts billing the server's owner, but understandably, that's made Hammer think twice about adding it.
Hammer and I talked about getting SRS up and running, and I think we're still considering doing so. Right now SRS is loaded onto the server, but it's not working for some reason and no one has had the time to figure out why. Hopefully, Hammer can take a look soon so that we can test SRS out, or I can do some research and figure it out. If you were to make this software, it could really be an enormous improvement to our DCS experience, giving us real impetus to use SRS, especially if your software also included a working ATC aspect. That would be amazing!
I think what you're intending to build here has indeed been done. The program is called OverlordBot 2.3.0, and it serves as an improved AWACS for DCS. It works with SRS, and the videos on youtube suggests it works fairly well. The problem with it is that after a certain amount of usage by players, the server owners are charged a fee. I think it takes quite a lot of request to the bot by players before the company starts billing the server's owner, but understandably, that's made Hammer think twice about adding it.
Hammer and I talked about getting SRS up and running, and I think we're still considering doing so. Right now SRS is loaded onto the server, but it's not working for some reason and no one has had the time to figure out why. Hopefully, Hammer can take a look soon so that we can test SRS out, or I can do some research and figure it out. If you were to make this software, it could really be an enormous improvement to our DCS experience, giving us real impetus to use SRS, especially if your software also included a working ATC aspect. That would be amazing!
Re: AI GCI
Thanks, Grifter. I was pretty sure something like this was out there. Didn't know it was called Overlord and didn't know they eventually bill the server owner. So, it may be something for me to continue looking into that does not depend on some sort of pay model. As for SRS not working on the server, I can try and take a look when I have time. That should be something very fixable.
Re: AI GCI
Id the overlord source is open, it may be possible to change out the Microsoft text to speech api to another that does not incur costs. I am pretty sure that is what can eventually start costing money.
also, some community server instances do run overlord, and have pretty strict rules to try and minimize the cost of running the software.
along these lines of discussion, has anyone tried the new built in voice chat? performs very much or exactly like SRS:
https://www.digitalcombatsimulator.com/ ... _Start.pdf
I am not sure why groups seem to stick with SRS though, it seems a lot of public servers do use it instead of built in. maybe someone can test/compare. i am pretty sure it is indeed enabled on the server. if it is decent, nd works well (enough), and has advantages over teamspeak then it is likely easier to implement than SRS.
also, some community server instances do run overlord, and have pretty strict rules to try and minimize the cost of running the software.
along these lines of discussion, has anyone tried the new built in voice chat? performs very much or exactly like SRS:
https://www.digitalcombatsimulator.com/ ... _Start.pdf
I am not sure why groups seem to stick with SRS though, it seems a lot of public servers do use it instead of built in. maybe someone can test/compare. i am pretty sure it is indeed enabled on the server. if it is decent, nd works well (enough), and has advantages over teamspeak then it is likely easier to implement than SRS.
Helmut
Re: AI GCI
I'm quite fine not using SRS. I don't know why other groups use it either. A lot of them have a discord and have channels for one team or the other. Like growling sidewinder. Plus I never hear anyone talking on SRS. On that server anyway. They all just do their own thing. Or they're in a team group chat in discord.
I like teams speak and it's working well I think for our group. We never really have more than eight on at a time.if we really wanted to have red versus blue we could open different channels to separate. Frankly also I like hearing the other call outs and radio traffic so I know what's going on elsewhere on the battlefield when I have my head down in the helicopter trying to blow up tanks.
I like teams speak and it's working well I think for our group. We never really have more than eight on at a time.if we really wanted to have red versus blue we could open different channels to separate. Frankly also I like hearing the other call outs and radio traffic so I know what's going on elsewhere on the battlefield when I have my head down in the helicopter trying to blow up tanks.
Re: AI GCI
Yeah I think though that the idea with SRS was if we did fly on another server and they require it.
v6,
boNes
v6,
boNes
"Also, I would prefer a back seater over the extra gas any day. I would have 80 pounds of flesh to eat and a pair of glasses to start a fire." --F/A-18 Hornet pilot
Re: AI GCI
So the idea of using SRS was that:
1: more realistic sounding comms
2: separate channels so that different flights aren’t talking over each other. Note that a common channel could still be active on second radio for broadcasting to all players when needed to communicate to whole package.
3: room for ICS channels for pilot and rio to have their own dedicated channel
4: can be used with software like Overlord for more realistic / accurate awacs and ATC
5: would allow RLG to “RAID” other PVE servers and fly as a group.
SRS is indeed superfluous but may provide an overall better experience. That said, if people don’t want to switch that’s fine.
1: more realistic sounding comms
2: separate channels so that different flights aren’t talking over each other. Note that a common channel could still be active on second radio for broadcasting to all players when needed to communicate to whole package.
3: room for ICS channels for pilot and rio to have their own dedicated channel
4: can be used with software like Overlord for more realistic / accurate awacs and ATC
5: would allow RLG to “RAID” other PVE servers and fly as a group.
SRS is indeed superfluous but may provide an overall better experience. That said, if people don’t want to switch that’s fine.
Re: AI GCI
We use SRS on our MVP Korea 1952 server. They pretty much use only 4 frequencies, 2 for each side. One is used for the home air base and the other is used for combat. Of course there's nothing stopping players from using other frequencies at any time, but typically you want to be turned to the common freq for your side so you know what's going on with others on your side. Keep in mind it's a little harder to change the radio frequency in the F-86 and the Mig-15. There are no standby channels!
SRS has some cool features you might not be aware of... Radio transmissions are effected by your altitude. If you are low or behind mountains, your radio coms won't be received by someone far away or on the other side of the mountain. I also like that SRS sounds more like a real radio when receiving transmissions.
Anyway, I started looking at the source code for Overlord last night. There's some pretty heavy complexity and code there to support what they do. There's a dependency on the Microsoft Azure cloud for voice generation - that's where the cost comes into play. In order to set it up, you have to provide an Azure cloud account. Note that it's not the developers you are paying, it's Microsoft you are paying (for the compute for the voice generation).
At any rate, I'm not sure I'm going to go any further with this. It might be nice to have an alternative to Azure for voice generation - so that's something that maybe could be done by somebody as an enhancement. I'm not sure I'd have to the time to do this. This was more of a "wouldn't be cool if..." and someone's already done it, which is great.
SRS has some cool features you might not be aware of... Radio transmissions are effected by your altitude. If you are low or behind mountains, your radio coms won't be received by someone far away or on the other side of the mountain. I also like that SRS sounds more like a real radio when receiving transmissions.
Anyway, I started looking at the source code for Overlord last night. There's some pretty heavy complexity and code there to support what they do. There's a dependency on the Microsoft Azure cloud for voice generation - that's where the cost comes into play. In order to set it up, you have to provide an Azure cloud account. Note that it's not the developers you are paying, it's Microsoft you are paying (for the compute for the voice generation).
At any rate, I'm not sure I'm going to go any further with this. It might be nice to have an alternative to Azure for voice generation - so that's something that maybe could be done by somebody as an enhancement. I'm not sure I'd have to the time to do this. This was more of a "wouldn't be cool if..." and someone's already done it, which is great.
Re: AI GCI
Search for MVP in the server list, and pick the one that says Korea1952. But it's restricted to planes that existed during the Korean War. Obviously the Mig21 was not invented yet.
The Korea1952 server is run by my Australian friend. They have a dedicated Discord just for that server, and it can be a pretty popular server at times. Discord is here:
https://discord.gg/KN5gRqcD
Oddly enough, I usually fly the Mig15 when I am on that server.
Re: AI GCI
Thanks for the info. I have been looking for a reason to get the mig15. Perhaps when it's on sale.