DARPA wants algorithms that filter text for intel analysis

DARPA wants algorithms that filter text for intel analysis

defensesystems.com

According to a May 22 broad agency announcement, the agency's Deep Exploration and Filtering of Text program calls for the development of algorithmic capabilities to do the following:
- See through language to meaning in text.
- Make use of key information contained in text documents.
- Cue up information sources that contain new developments for analysts.
- Automate the initial stages of report writing.

Proposals are due by July 10.

(visit the link for the full news article)


Related News Links:
www.fbo.gov...

Presumably there will be takers for this bid proposal.

According to the documentation:

DARPA is soliciting innovative research proposals in the area of deep natural language understanding. The Deep Exploration and Filtering of Text (DEFT) program seeks to develop the ability to see through language to meaning in text, to make use of key information contained in text documents, to cue up information sources that contain new developments for analysts, and to automate the initial stages of report writing. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

I have never seen a proposal requesting "revolutionary advances" before... they must be willing to spend borrow a lot of money (in our names of course) to get it.

The request is noteworthy since we are clearly at the stage of implementing the big data sift now that there are no liabilities for spying on every citizen possible.

Department of Defense (DoD) operators and analysts collect and process copious amounts of data from a wide range of sources to create and assess plans and execute missions. However, depending on context, much of the information that could support DoD missions may be implicit rather than explicitly expressed. Having the capability to automatically extract operationally relevant information that is only referenced indirectly would greatly assist analysts in efficiently processing data. Automated, deep natural-language understanding technology may hold a solution for efficient processing of text information and for understanding connections in text that might not be readily apparent to humans.

Brace yourselves...for our accusers will now potentially be faceless machines.

Some of the challenges they expect to overcome is having a machine extract:

• People and groups
• Events (who, what)
• Geo-spatio-temporal information (where, when)
• Causal explanations (why, how)
• Facilities and equipment
• Modality and beliefs
• Anomaly, novelty, emerging trends
• Inter-relationships, entailments, coreference of entities and events
• Disfluencies/disjointedness
• Dynamic, perishable, changing situations
• Ambiguity and vagueness
• Uncertainty, inconsistency, contradiction
• Unstated, implied information

Imagine that! A machine that can "infer" ambiguity, vagueness, unstated information.. modality and beliefs!

The DEFT program aims to enable analysts to discover implicitly-expressed, actionable information. This aim requires the development of automated deep natural language understanding technology. Technology developed in DEFT is expected to provide the capability to identify and interpret both explicit and implicit information from highly ambiguous and vague narrative text, and integrate individual facts into large domain models for assessment, planning and prediction. All of these capabilities are expected to emerge from a range of different research foci in language understanding.

Interesting times... too bad we can't be free to communicate with one another about it... we might be said to be "implying" something "actionable."



defensesystems.com
(visit the link for the full news article)

reply to post by Maxmars


This is scary and slightly amusing. It's scary to me because now the spooks will be reading between the lines, so to say... Which really means that they can project any number of 'meanings' to simple text.

It's amusing to me because it means that 'THEY' are scared of what we may be thinking... Which is good and bad at the same time.

"calls for development" = wish list.. a machine that could understand natural language to the point where it could infer tone, symbolic meanings and "read between the lines" would be one of the holy grails of A.I. As far as I know, no-one's come even remotely close to such a feat yet and I suspect no-one will for quite some time.

If someone did find a solution I'm sure a google-bot-esque script wouldn't be too far off to trawl the internet looking for suspicious conversations (no more need for keyword searching if your bot is able to recognise speech). If it does ever happen then I guess Tor is going to suddenly become very very popular

reply to post by LordGoofus


I have found that DARPA rarely engages in "pie-in-the-sky" requests. I think we may be much closer to this than you may believe. But in either case... whether its achievable or not; I would have liked to have imagined it's development as a means to help people... not hunt them.

reply to post by Maxmars


Exactly.

Generally you can add a good 30 years of research and development ahead of what we see today commercially to anything the military, DARPA, or the clandestine guys are developing. Project Winterhaven was in the 50s if memory serves and that technology is now being used on machines like the B2 Stealth.

IMO, DARPA should be closely paid attention to by the interested public. Ten years ago they started developing exoskeleton 'cyborg' soldier tech, and now it's not far from being used in the battle field. Same thing for remotely operated drones. Soon we will be seeing them all too often.

reply to post by Maxmars


FYI - has all the earmarks of a hardwired bid.

reply to post by AlchemicalMonocular


I thought so too. The "revolutionary advance" term is a dead giveaway. I bet it comes straight from someones' PR department.

We shall see... I suspect I even know the nationality of the soon-to-be "accepted" bid.

reply to post by Maxmars


This may be a huge jump to make the link but... could this be what the UK GCHQ officer, Gareth Williams was working on, while seconded to MI6 and killed in the "body in a bag" case?

A talented pupil, he graduated from Bangor University with a first class degree in maths aged 17 after beginning his university studies while at secondary school.

Mr Williams went on to study for a postgraduate certificate in mathematics at St Catharine's College, Cambridge, in 2000, but dropped out a year later before taking up the job at GCHQ in Cheltenham.

His boss there has spoken of him as a "world-class" intelligence officer and "something of a prodigy".

Mr Williams returned to his flat - half a mile from MI6 headquarters on the banks of the River Thames - on Wednesday 11 August 2010 after a fly-drive holiday to the west coast of the US.

Police believe that Mr Williams, whose family think may have been killed by an agent "specialising in the dark arts of the secret services", was helped into the bag.

That is a "You've got to be kidding" thread headline.

Consider that they have had facial recognition software and computer power to "read" thousands of faces in a crowd of football fans or protestors for years. Then it stands to reason that those programs are vastly more complex than analyzing a few thousand words put into electronic form. They've had that capability long ago with the carnivore program from the 1980s. It was in a simplier form back then but you can bet later versions are smart enough to discern when you are talking in code about something.. Again, such programs are far less complex than other complex algorithms that look for suspicious movements of individual in airports, etc.

You can run, but you can't hide these days. That all-seeing on on the US dollar bill is becoming more apt every day. You could think that the government is almost like God.

I have never seen a proposal requesting "revolutionary advances" before... they must be willing to spend borrow a lot of money (in our names of course) to get it.

It's not going to happen due to a very fundamental flaw in human contextual reasoning in those interpreting data expecting to find a problem and similar predispositions in the eventual coders.

The problem is based on the knee jerk reaction to the information age; that people of varying educational backgrounds are going to go hell bent rampaging because their expectations don't match reality. More likely, because the reality of basic Big Media communication (on politics for example) is to use psychological cues to invoke reactions. The internet is now a forum for the venting of those invocations and some of the danger this coding presumes to prevent has been shown to be created through FBI intereference/suggestion,

My point?

There is no absolute standard for applying a static formula to context.

This is interesting to say the least. Although security services have had access to high powered computer programming for some time, and have created and used extremely advanced imaging software packages to identify indivuals in crowds, that is a significantly easier task than essentially asking a computer to make qualitative judgements about the content of a transmission.

We are essentially at the point where DARPA has requested a computer that renders MEANING! Have you ANY idea just how difficult that would be to organise? We humans can form opinions about things, because we have reasoning capacity which automatically derives a meaning from everything that we do, or say. From everything we see and hear, touch and feel. We spend all day processing the stimuli we are exposed to using the tools we have which have evolved over time uncountable, since we were little more than tree dwelling rodents.

Call it soul, call it self awareness, call it whatever you will, but organic life is the only thing on this Earth that has ever derived meaning from a scentence, be that written word, spoken word, or barked pack communication between dogs. Computers may be able to assess wether or not certain words are spoken, or certain phrases used in chat logs, maybe even phone conversations if correctly programmed, but as for them being able to discern actual content value and summarise the key features of a given text, speech, or document of some kind. This would be a whole new dawn for computer science.

The reason I say this is simple. In order to discern meaning, one has to understand not only the presence of a word, but its wider meanings. I can read the scentence "My God, this heat wave is horrible", but in order to understand the meaning behind it, I have to think about what it means for me, for the condition of the water table in our region, the number of extra washloads that will be washed because of the increase in sweat, and so on and so forth. This would not be a simple task of databasing a bunch of information and building a program that connects it all together. To do that would be like secret google. It would be crap and give them spam all day long. Oh no. DARPA are after a machine with artificial intelligence on a scale that simply does not exist in this day and age. Only intelligence marks the difference between a mere computer, and a truely thinking machine. And to do the work it is intended for, it will have to be a very capable linguist, wordsmith, conversationalist, and familiar with every local dialect and slang term, in every language, in every culture, sub culture, and club in the whole damned world. Not only that, but it will have to be able to identify the implications of words, not just the words themselves.

What they are talking about is... well for Star Trek fans I would say it is the difference between Data and the Enterprise computer in The Next Generation in terms of thier capacity for learning and undertstanding.

reply to post by Maxmars


These guys just dont let up do they?
I would presume this means those "AI" life forms they are
working on just are not up to running speed yet.

I have a few thoughts...."pork" and "cloud".
hmmm.....

Originally posted by W3RLIED2
reply to post by Maxmars

 

This is scary and slightly amusing. It's scary to me because now the spooks will be reading between the lines, so to say... Which really means that they can project any number of 'meanings' to simple text.

It's amusing to me because it means that 'THEY' are scared of what we may be thinking... Which is good and bad at the same time.

The various combinations of words is statistically important to meaning. This they know.

Really, it's too late to be scared, go with amusement.

Originally posted by W3RLIED2
It's amusing to me because it means that 'THEY' are scared of what we may be thinking... Which is good and bad at the same time.

I disagree. It means that they are concerned with accuracy...which is a good thing. If someone I don't know is going to be looking at my opinions...I would like them to interpret correctly...I want to disagree with things and not have people think I a terrorist...indeed, I dislike that I even have to feel the need to clarify such things...

They should use vedic mathematics, im sure the key to this math equation is in the 16 sutras.

Basically it is an algorithm that takes a large text and generates a human readable summary that is all it is.

Originally posted by THE_PROFESSIONAL
Basically it is an algorithm that takes a large text and generates a human readable summary that is all it is.

Summed up in one word?

Context.

Originally posted by Maxmars
reply to post by AlchemicalMonocular

 

I thought so too. The "revolutionary advance" term is a dead giveaway. I bet it comes straight from someones' PR department.

We shall see... I suspect I even know the nationality of the soon-to-be "accepted" bid.

Hmmm. Should it go to the cloak to the US's dagger? Or perhaps to a country better-suited toward hackable back doors? Could we somehow find both in one country?

Overall, it seems DARPA is anticipating the same exact phenomenon I'm anticipating: as the internet and all communications become more monitored and analyzed, a new language, common among a cohort's members, will emerge. It will erase conventional meanings and replace them with meanings known only to that group. DARPA is preparing for the possibility that one day "I prepared roast beast for dinner" will actually mean "Coup meeting at five over at Barney's place". Just wait - people hate being controlled and when they are, they often manage to appear as if they're being controlled until one day, those in control are schooled. It's already begun, and it's spreading...

Well, just to add my 2 cents worth:

Former DARPA chief Regina Dugan left DARPA and joined Google, now that is telling!

www.popsci.com...

Puzzling?